All of the universe which is observable, we can see :)
But you're right---there's lots of additional universe out there that we can't see, and we'll never be able to. In fact, because of expansion, more and more of the visible universe is actually leaving the region which we can see---which is called our 'light-cone'.
Star with a point in space-time---called point 'A'. Now imagine light traveling away from that point in all directions. In the plot below, time is graphed on the vertical axis, and space is in the horizontal plane. Every second that goes by, the light goes 1-light-second (ls) further away. If you trace out the path of those light-rays, it defines two cones:
Everything in the cone behind point A is in its 'past', everything in the cone in front of it is in its 'future'. Everything outside of both cones is 'causally disconnected' (like point 'E'). Points outside of the cones will never interact with point 'A' (because information from them is limited to the speed of light).
If you imagine that you are at point 'A'. Then the visible universe is everything within your past-light-cone. The circle at the bottom would be the border ('event horizon') of your visible universe.
Because space-time is expanding, the shape of the 'cones' is actually bent, kind of like this:
But anyway, there is lots of universe outside of your light-cone (outside of out 'observable universe'), possibly infinite amounts of it (we don't know).
This is a rather lengthy answer as I tried to go a bit in depth; there is a short summary at the end.
Will we see more or fewer stars with time?
The short answer to this is: We see less stars with time, due to the fact that cosmic expansion is accelerating. Although what we really see at the relevant distances are galaxies; single stars are far too far away to be resolved.
The first thing to realise to understand this is that the speed limit of special relativity doesn't apply to cosmic expansion. If you imagine the galaxies as raisins in an enormous, rising dough, what SR tells you is that nothing can move through the dough faster than light. But if expansion is the same everywhere, and the dough is large enough, then an arbitrarily small speed at which the dough would rise would make all raisins beyond a certain distance recede from your local raisin faster than light. This "certain distance", in the Universe, is called the Hubble Distance. Galaxies farther away from the Hubble Distance are receding from us faster than light.
This, however, is not the limit beyond which light cannot reach us. If a galaxy is receding from us faster than light, a photon emitted from said galaxy towards us will initially be receding from us, but it will crawl to regions of space - at local light speed - which are receding from us a bit slower than the one from which it was emitted, and so it will recede a bit slower from us than before, etc., until it finally enters a region of space that is receding from us slower than light, from which point it is trivial that it will eventually reach us.
This is not always the case, though. There is, in certain cosmologies (including our own with an accelerating expansion), a distance,from beyond which a photon emitted will never reach us. This happens if the Universe expands faster than the photons can traverse it. This distance is called the cosmic Event Horizon, and while it expands in terms of absolute distances, it is shrinking in terms of co-moving distance. The co-moving distance is the distance measured in a coordinate system that expands along with the Universe - such a system will have galaxies more or less sitting in the same coordinates throughout the history of the Universe, and so the Event Horizon having a shrinking co-moving radius from us directly tells us that fewer objects will be inside this radius in the future.
The way we can in theory observe these distant galaxies leave our event horizon - and the reason why it has its name - will be very much like we would observe an object falling into the event horizon of a Black Hole: relativistic effects will redshift the light emitted from such an object indefinitely, which also slows down the time as observed by us. So the object will appear to slow down asymptotically to a complete freeze, while the light will be redshifted towards infinity and become undetectable by us, although you could argue that it will never, in theory, disappear completely. The last we will ever see of such a galaxy is a point in its history (that is not special to the galaxy itself), after which we will never receive more recent information about the galaxy - this is why it is called an event horizon.
Can stellar objects go outside of the scope of the observable universe or is it only growing with time?
This is not easy to answer with a clear "yes" or "no", because the term "the observable Universe" is not clearly defined.
The event horizon that I mention above is defined as:
Event Horizon: The distance, from beyond which a photon emitted towards us now will never reach us.
And we just established that this distance is shrinking in co-moving coordinates, so yes, galaxies are escaping our cosmic event horizon as we speak.
But we can see that the Event Horizon has the property that is is a cut-off in time, as well as Space: For any galaxy, it is the last event in its history that we will be able to observe. But we could also ask a different question:
What is the farthest distance, at which we can see the first event in an object's history?
Or in other word, how far can a photon emitted at the time of the Big Bang, have travelled today? At which distance are we observing the Big Bang? We call this distance the Particle Horizon:
Particle Horizon: The distance from which a photon emitted during the Big Bang will reach us now.
We normally use the Cosmic Microwave Background as a proxy for this; it currently has a redshift of ~1100 and a proper distance of 46 billion light years from us (remember, that is the current size of that space, but it has expanded continuously while the photon was travelling, so the photon will still have travelled only the 13.8 billion light years we would expect from the age of the Universe). While the event horizon is growing in absolute, but shrinking in co-moving coordinates, the Particle Horizon is growing in both coordinate systems, so that horizon is expanding forever.
Summary
To answer this question properly, we have to think in terms of both time and space. On one hand; for all galaxies, there is a point in their history that is the last we will ever observe, so in that sense, galaxies (and thus stars) are receding out of the "observable Universe" as we speak.
On the other hand, the distance at which we can see the first events in the Universe is expanding and always will be. But the time limit on how much of the history of these new regions we can study before they slow down to a freeze and redshift to infinity will get ever shorter.
PS: The math behind
There is an article that is relatively easy to read for someone with a background in physics, written by someone way smarter than me, on ArXiv.org, explaining this whole thing in better theoretical detail.
Best Answer
As it happens this issue has just been mentioned in the Science Fiction Stack Exchange. The simple answer is that we cannot know that there is anything outside the observable universe. The best we can say is that it seems likely.
We approximate the universe using the a spacetime geometry called the FLRW metric. This is based on the assumption that the universe is the same everywhere - technically that it is homogeneous and isotropic. If the universe is the same everywhere then obviously it's the same beyond the bits we can see, but remember that this is just an assumption.
It's possible to construct a universe that looks like an FLRW universe locally but contains no matter outside some boundary. This metric is called the Oppenheimer-Snyder metric and was devised as an approximate description of a collapsing star forming a black hole. However we can reverse the time direction and the metric then describes matter emerging from a white hole. As long as the boundary is farther away than the edge of the observable universe we would not be able to tell the difference between an Oppenheimer-Snyder universe and an FLRW universe.
However the Oppenheimer-Snyder construction is a rather artifical one. It is created by welding together a patch of spacetime described by the FLRW metric and a patch described by the black hole (Schwarzschild) spacetime. While it's technically possible for this geometry to describe our universe it would take a deity with a rather peculiar sense of humour to arrange the universe in this way. The FLRW universe seems much less contrived and therefore more likely.