One has to distinguish the two frameworks: the classical, light; the quantum, photons.
The classical electromagnetic wave, of which visible light is a part of the frequency spectrum, emerges out of zillions of photons, the quantum of light. This happens because the photon has an energy E=h*nu, where h is the Planck constant and nu the frequency of the classical wave that will emerge from zillions of photons . It also has a spin and can be described by a quantum mechanical wave function that allows the build up of the classical wave from the quanta. The energy of the classical wave is the addition of the individual photon energies building up an amplitude.
Is the sine wave just a way to represent the periodic change in field strength, or do the fields occupy a volume such as would be generated by rotating a sine wave about its axis?
The sine wave and cosine wave belong to the classical electromagnetic wave. Not to individual photons. Individual photons have a probability of appearing in space that is described by a sine/cosine wave, but not a distribution of its energy in space time. The photon's energy is one whole quantum. The emergent classical wave has a sinusoidal energy distribution in space time of the same frequency .
Does a light quantum have length, or is it only an instantaneous value at a point in space (and then how can it be red- or blue-shifted)?
The photon has no length, it is an elementary particle . The shift in frequency that is assigned a change in color is an emergent effect from the zillion of photons. For the individual photon, a red shift means a lower frequency/energy h*nu, a blue shift a higher frequency/energy h*nu.
How does the magnetic field component satisfy the requirement that all field lines be closed?
The photon does not have a magnetic field component, it is characterized by the potential entering Maxwell's equations and it will build up the magnetic and electric field components of the emergent classical wave. By itself it is a particle characterized by a probability distribution for its space time location.
Does light even behave like this on a discrete level?
On a discrete level, in the double slit single photon experiments which show interference patterns by individual photons, i.e. the probability distribution manifesting in the build up of the experiment, the emergence of the classical wave that coincides with the quantum frame is displayed clearly.
When a wave travels through a rope, the rope goes up and down, the position of all the 'rope-particles' changes, they oscillate and this makes up the wave.
With light, it is indeed the electromagnetic field oscillating, but you shouldn't think of the arrows that represent that field in your first picture of light as 'extending into the rest of the space'. They are not spatial, they just represent the magnitude and direction of the electric/magnetic field at that point.
Best Answer
If the person drawing the graph bothers to label the axes you'll see that the thing that "goes up and down" is not displacement as it is in a wave on a string but electric field strength.
So, no, nothing is moving off the line of the ray, but the because electric field is a vector the oscillation does have a direction associated with it (and therefore polarization makes sense).