If by "yellow light", you mean a light with wavelength between 560 and 590 nm, and by "blue paper", you mean paper that reflects light only if its wavelength is between 450 and 490 nm, then yes, if the only illumination for a piece of blue paper is yellow light, it will appear black. But "yellow light" is generally light that has a mixture of wavelengths such that the average wavelength is between 560 and 590 nm, not light such that every photon has a wavelength in that range. That's why red and green light together looks yellow: when our eyes detect both red and green light, our brain perceives it as yellow.
Similarly, most "blue" paper does not reflect just blue light. Rather, it reflects light most strongly in the "blue" range of wavelengths, and/or reflects light whose wavelengths average out to be in that range.
So when mostly-yellow-but-also-has-a-little-bit-of-other-wavelengths light hits mostly-reflects-blue-but-also-weakly-reflects-other-colors paper, some light will be reflected. What color it looks like will depend on the exact composition of the light and the exact reflective properties of the paper. In the example you gave, the pepper reflects a significant amount of red light, but still looks darker than the red parts of the playing card. In green light, the pepper is bright green, while the red parts of the playing card are almost black. This implies that the pepper reflects green light more than it reflects red light, but still reflects some red light, while the red parts of the playing card reflects very little green light. This makes sense: the pepper is a natural object, while the playing card has an artificial dye specifically designed to be a particular color.
Best Answer
In a Newtonian/Galilean world, where $c$ is infinite, you could not escape Olbers' paradox with an infinite universe. Any line of sight would eventually intersect the surface of a star, and so the whole sky would be as bright as the Sun. This is true whenever two hypotheses are satisfied:
The first condition says all lines of sight terminate on stars. The second says that we see that stellar surface, whether we have to wait an arbitrarily long but finite time for it to get to us (because $c$ is finite, but we have an infinite past during which light traveled), or not (because any finite distance is covered instantaneously, so it doesn't matter how long the universe has been around).
Note by the way that infinitely quickly propagating influences and infinite, homogeneous universes don't mix well at all, not just with regard to light. For instance, the gravitational effect on us by an infinite, uniform distribution of mass is undefined in Newtonian cosmology. So even before Olbers, Newton knew something had to give if one wanted an infinite universe.