There are two kinds of circular polarizers: right (clockwise) and left (counterclockwise). The first will transmit right circularly polarized light and absorb left circularly polarized light; the second does the opposite.
Turn the glasses over so the light goes the other way, and you will see that the effect is different.
A quarter wave plate, as @Jasper pointed out, is always wavelength (and light propagation angle) dependent. Most polarizers work less well with shorter wavelengths.
If light is randomly polarized, then passes through a linear polarizer, it becomes linearly polarized. Then when the light subsequently passes through the quarter-wave plate, the linear polarization converts to circular polarization. A quarter wave plate has two axes: a "slow" axis and a "fast" axis. Light propagates a little bit slower if it is polarized in the "slow" direction. The thickness of the quarter wave plate is precisely enough to slow light polarized in the "slow" direction, by a quarter of a wave (relative to light polarized in the "fast" direction. If you draw the waves, you'll see that the QW plate does nothing to light that is polarized precisely along either axis; but if light entering the QW plate is polarized at 45 degrees it comes out circularly polarized in, say, the clockwise direction. Rotate the QW plate by 90 degrees and the light will come out circularly polarized in the counterclockwise direction instead. If light entering So, the orientation of the QW plate relative to the linear polarizer is crucial, but when light enters from the linear polarizer side everything works right because the linear polarizer absorbs the portion of light that's polarized "wrong".
On the other hand, when randomly polarized light enters from the QW plate side, essentially nothing happens to it because it is still randomly polarized when it exits the QW plate. When it subsequently enters the linear polarizer, the light simply comes out polarized in the direction determined by the orientation of the linear polarizer.
Now consider what happens if circularly polarized light passes through from either side. If it enters from the QW plate side, *and if the handedness (clockwise or counterclockwise) is right, the QW plate converts it to linearly polarized light that passes almost 100% through the linear polarizer. If the handedness is wrong, the QW plate converts it to light that is linearly polarized light but in the wrong direction so it gets nearly 100% absorbed by the polarizer. Rotating the polarizer makes no difference to the amount of light that passes through, but it affects the polarization angle of the exiting light.
However, if circularly polarized light enters from the linear polarizer side, the polarizer absorbs the component that's not aligned with the polarizer, then converts the remainder to circularly polarized light with a certain handedness. Rotating the circular polarizer in this case makes no difference.
The third case to consider is that of linearly polarized light entering the filter from the QW side or linear polarizer side. If linearly polarized light enters from the QW side, the QW plate converts it to elliptically polarized light: circularly polarized if the linearly polarized incident light is at 45 degrees to the axes of the QW plate. The linear polarizer then transmits the portion of the elliptically polarized light that has the proper linear polarization. If linearly polarized light passes first through the linear polarizer, then through the QW plate, the linear polarizer passes only the fraction of the light having the proper linear polarization (note: linear polarization can be treated as a vector, so light polarized at 45 degrees from the vertical is equivalent to two vector components polarized vertically and horizontally, each with amplitude $1/sqrt(2)$). The QW plate then converts the passed light into right- or left-circularly polarized light.
There is a really good Harvard lecture here about polarization. Lots of good graphics in the Wikipedia article on circular polarization.
You will get color effects whenever different wavelengths are affected differently. In your experiment, you found that rotating the 3D glasses 90 degrees to the screen's vertical axis changed the color of light passing through the glasses. That means that the light emitted by the screen is polarized differently for different wavelength ranges. You don't know without further experimentation, in what way the polarization differs by wavelength. However, if you turn your 3D glasses over, they will act effectively as linear polarizers. You can explore the linear polarization of different wavelength ranges by putting colored cellophane between the polarizer and your eye as you rotate the polarizer while looking at the screen.
To see some really beautiful effects, look at some wrinkled cellophane tape between two crossed linear filters.
Temperature, colour and energy all mean different things.
A single 'photon' has an associated energy, but it's misleading to think of this as a colour.
Three things that are (for most intents and purposes) equivalent, are energy $E$, wavelength $\lambda$ and frequency $f$, which are related by
$$ E = hf = hc/\lambda$$
where $c$ is the speed of light and $h$ is Planck's constant. Note that as the wavelength gets smaller, $E$ gets bigger.
Humans have three kinds of cone (colour sensing) cells - red, green and blue, but these do not "only" see red green and blue respectively - they are excited to varying degrees by the other wavelengths as well.
To a human, a combination of 500nm and 660nm light appears yellow, but this is physically a different situation to e.g. a yellow sodium lamp, which emits primarily ≈580nm photons.
Light from something like the sun or an incandescent lamp is even more complicated - these are near-perfect blackbodies, meaning that they emit at all wavelengths to some extent:
This excites all three of our cones equally, which our brain processes as "white". The crucial point is that there is no such thing as a "white" photon, but there is such thing as a "blue" photon.
The changing colour of a blackbody as it gets hotter is due to the movement of the "peak" wavelength, i.e. the wavelength at which most light is emitted, which for the Sun is about 500nm. If the sun were to get hotter, the peak would shift to the left, exciting more blue cones and fewer red cones. Likewise, if it got colder, the peak would shift to the right (lower energy), making the Sun look redder.
The Bottom Line
All wavelengths of photons carry energy, "white" light is a mixture of many wavelengths, and filtering this mixture to only get e.g. red light will make the filter heat up / reflect the energy elsewhere.
Best Answer
The size of a photon means something entirely different than it would in classical physics. A photon is described by a wave function. From the wave function, you can calculate the probability of finding the photon at a particular position if you measure it. You measure it by interacting with it, which changes the photon.
One way a photon interacts when it hits a screen is to promote an electron in a single atom to a higher energy orbital. The photon is absorbed and ceases to exist. It does not matter how big the photon's wave function is. It will interact with just one atom this way.
Other ways of interacting can be more spread out. A mirror is a smooth metal surface. Conduction electrons in a metal do not stick to a single atom. They spread out. A photon interacts with the sea of spread out electrons. The result is a photon just like the incoming photon, but reflected away. This interaction is easiest to understand from a classical description. Classical light is an oscillating electromagnetic field. It causes surface electrons to vibrate, and the light is absorbed. A plane of vibrating electrons emit light. This creates the reflected photon.
For more about the nature of a photon, see How can a red light photon be different from a blue light photon?.
Also see Does the collapse of the wave function happen immediately everywhere?. This describe an electron, but it works the same for a photon.