Yes all the stars you can see belong to the Milky Way galaxy. The Sun is located about two-thirds of the way out from the galactic center. Thus the band across the sky known as the "Milky Way" (from whence the galaxy name came) is the main disk of the galaxy. However, all the individual stars we see are still inside that disk, we just happen to be closer to them so that they stand out as individuals.
The galactic disk has a thickness to it (about 1000-2000 light years) so the closer stars are not all in the band we see as the main disk simply because they are closer and the relative angles between the sun and these stars puts them "up" higher than the main galactic disk as seen from Earth.
This is a simple and clear issue, with a unique answer. I see other replies mentioning weather conditions, dark adaptation and so on. That's just so much hand waving, given that the first thing you said was "I've always lived in somewhat large cities".
The core problem here, by a very wide margin, is light pollution if you live in a large city. This is the one factor, above everything else, that affects your ability to see the stars.
Here's a light pollution map:
http://www.jshine.net/astronomy/dark_sky/
The white zones are the worst, and they are in the middle of the cities. Black zones are the best.
Here's a somewhat better (but not perfect) comparison of a dark sky versus light polluted sky (your picture was taken with a very long exposure that doesn't look very realistic):
The dome of light above the city is very visible if imaged from afar:
Long exposure pictures in cities will reveal the orange skyglow, which is the main reason why you can't see the stars - it's like noise masking off the faint light from the distant objects:
Light pollution affects primarily the observations of faint objects, such as nebulae or distant galaxies. Bright objects such as the Moon, the big planets, or some of the bright stars, are not affected by light pollution.
Using a telescope with a large aperture alleviates the effects of light pollution to some extent, but it cannot work miracles. A dark sky is always better.
Usually a 1 hour drive away from the city will bring you in a place with dark sky, free of light pollution - but it depends on several factors. In such a place you should be able to see the Milky Way with your naked eye. The Andromeda galaxy also is visible with the naked eye if the sky is dark enough.
Best Answer
With the assumptions you have added to the question the answer would be that every ray would end on a star and that the night sky would be as bright as the surface of the average star - i.e. quite bright.
You are essentially describing Obler's paradox. Per Wikipedia:
With the questions assumptions of a "snapshot" of the universe and that light travels at infinite speed along with the additional assumption that the universe is infinite,the result would be a very bright sky. Now the measured curvature of the universe is flat to within about 1%, so if the universe is in fact flat or even negatively curved the universe would be infinite and you would obtain Oblers' paradox. If it turns out that the universe is positively curved it would be finite but you would still have Oblers' paradox since the ray that "circles" the universe without hitting a star would keep circling the universe until it does hits a star.
The inverse square law does not diminish the brightness since the number of stars at a given distance increases as the square of the distance, so the two factors cancel each other out.
The reasons our night sky is not as bright as the surface of the sun is:
There is one other reason why the sky should be as bright(er) than a stars surface. Up until 380,000 years after the Big Bang, the entire universe was filled with a very hot plasma. At 380,000 years it had cooled enough for the electrons and protons/nuclei to recombine and produce neutral hydrogen and neutral helium etc. At that time the universe became transparent and was filled with the photons from that the hot ionized plasma. However since then the continued expansion of the universe has redshifted those visible light photons down into the microwave range. That is the source of the Cosmic Microwave Bacground (CMB) radiation that now has an effective temperature of 2.7 degrees Kelvin. However, there are approximately 1,000,000,000 of those photons per proton in the universe so the sky is still quite bright, just not in photons that our eye can see.