One half of the problem is light. Light is an oscillating electromagnetic field. The frequency of oscillation determines the color. Higher frequencies have shorter wavelengths and are blue. Lower frequencies have longer wavelengths are red.
Light is a mix of a range of wavelengths. Sunlight contains a wide range of wavelengths, including some that are too long or short for us to see. A laser contains a very narrow range.
How light interacts with matter is the other half of the problem. Electromagnetic fields exert forces on electrons. Electrons in atoms or molecules are constrained in how they can move.
Metals have electrons that can move freely. This makes them good conductors. Freely moving electrons reflect light. Light makes electrons vibrate and absorb the light. Vibrating free electrons emit light and stop vibrating. Thus metals are shiny.
But even in metals, electrons are constrained. For example in copper, electrons can vibrate more readily at low frequencies. Copper reflects red light better than blue. It reflects better still in the infrared and even worse in the ultraviolet. Here is a graph from a random website, http://www.minoanatlantis.com/Minoan_Mirror_Web.php
So if you shine a red laser on copper, it reflects almost all of it. If you shine a green laser on copper, it would reflect less.
Other materials have different ways of interacting with light. But at each wavelength, they may reflect some and absorb some. They reflect more at some wavelengths and less at others. Even black materials reflect some light.
The highly interesting thing is, brown absorbs all other colors but reflects red and green.
This may be a source of your misunderstanding.
The way humans perceive color is by having three types of photoreceptors in their eyes. Each of them has a broadish response function which means, that the "green" receptors react well to pure green light (as in 555nm), but also are somewhat excited by yellow (589nm) and hardly respond to red (>600nm). The same reasoning goes to the "red" receptors, which respond well to red (632nm, for example), but the yellow (589nm) also affects them somewhat.
Therefore, when you shine a red and green light onto a white screen, it reflects both red and green photons, exciting both "red" and "green" photoreceptors in your eye, which the brain interprets then as yellow. But similarly, you can simply illuminate the screen with a yellow light and get a 589nm wave, to which both mentioned types of receptors are sensitive. As a result, they're both excited and the brain again interprets (this time correctly) this as yellow. (see first figure)
Now the way pigments mix is slightly different. A pure pigment absorbs all light, except light of its color. Again, not perfectly, but with a relatively broad response, i.e. a green pigment may reflect some blues and yellows, but the main part is centered on the green. Similarly, a red pigment absorbs all, but red. So when you mix the pigments, qualitatively you're summing up their absorption functions. The presence of red prevents you from seeing much of the reflected green light, conversely, the presence of green prevents you from seeing much of the reflected red. However, in between, where the yellow lies, their response functions overlap and they reflect (although strongly suppressed), some yellow.
This dark yellow, smeared over a wide portion of the spectrum is perceived as brown. (see second figure)
Fig. 1
Brain identifies "red" and "green" receptors to both be strongly excited in roughly equal parts. It figures, the wavelength of incident light to lie at the point where they are equally sensitive and concludes, that incident light is yellow. However, it may just as well be equal amount of red and green photons.
Fig. 2
Red and green pigments have - qualitatively - these kind of reflection spectra. When you mix them together, you get the worst of both worlds: neither green, nor red is reflected particularly well, some residual reflection remains in between.
Best Answer
It's worth emphasizing something that you say: "As far as our eye is concerned, there is no difference between 'yellow' and 'red + green'". The point is that our eyes perceive these distinct physical phenomena to be the same. So there are multiple ways things might work out.
Yellow is a color of its own, independent of red, and independent of green. Wikipedia tells me that yellow is a color of light with wavelengths between roughly 590 and 560 nanometers. However, the excellent link @anna v points to explains that the human eye might perceive different combinations of light (with different wavelengths) as being the same color. In this case, it is possible to stimulate the human visual cortex to perceive yellow with a single wavelength of — say — 575 nanometers. But it is also possible to stimulate the human visual cortex to perceive something we experience as yellow with two different wavelengths of — say — 675 nm and 540 nm at the same time.
So an object illuminated with white light may absorb every wavelength but 575 nm (including red, green, and blue), or it may absorb every wavelength but 675 nm and 540 nm (including yellow and blue). Or maybe it'll absorb only wavelengths below 520 nm (so just blue). Our eyes can't tell the difference.