For a given volume, light things float and heavy things sink. The cup sinks when you fill it with water because it becomes heavier, and therefore more dense. When the cup becomes more dense than water, it sinks.
The cup would sink just as well if you filled it with rocks, lead, etc. The condition for the cup to sink is that its weight must be greater than the weight of the water it displaces (i.e. its weight must be greater than the weight of a cup exactly the same size, but made out of water and filled with water.)
You can read a number of other questions on similar material here: https://physics.stackexchange.com/questions/tagged/buoyancy
You would think that's an easy question, but it's not! Actually many things involving fluid mechanics are far harder than they seem. Anyhow a team of scientists at the University of Lyons in France have been working on this. See here for their paper or here for a more user friendly version.
Water has a tendency to stick to glass, so the water in your glass (or in the example above your teapot) has to detach from the glass at the lip to avoid dribbling. If you pour fast the momentum of the water will pull it free of the glass and it will pour cleanly. If you pour slowly enough it is more energetically favourable for the water to remain stuck to the glass and it will flow over the lip and down the outside of the glass.
In the paper above the scientists recommend controlling the wetting properties of the teapot to reduce the adhesion of the water to the spout. You could try dipping your glass in fabric conditioner as this will hydrophobe the surface. Also a sharp edge means the water has to change direction fast to dribble, so it will reduce the flow rate at which dribbling starts.
Response to Olly's comment: to do this with any degree of accuracy you need to reach for your finite element software and numerically solve the Navier-Stokes equations. But let's have a go using a very simplified model. NB I'm making this up as I go along, so you should probably check it before sending it to the Nobel committee.
Consider water flowing off an edge; like a river flowing over the edge of a cliff, but on a teapot sized scale. The diagram below shows the geometry.
The water is flowing at a velocity $v$ through a channel (i.e. spout) of width $l$ and depth $d$.
Where the water leaves the channel and flows out into the air you are exchanging a water/teapot interface for a water/air interface. let's call the water/teapot interfacial tension $\gamma_{wt}$ and the water/air interfacial tension (aka the surface tension) $\gamma_{wa}$. The units of $\gamma$ are force per unit length i.e. if you draw a line of some length $\ell$ the force normal to that line is $\gamma\ell$.
If you look at the point where I've drawn the force $F$ acting,i.e. where the water leaves the edge, then the force $F$ acting in the direction I've drawn (opposite to the velocity) is:
$$F = (\gamma_{wa} - \gamma_{wt})\ell $$
where the length $\ell = l + 2d$. A positive value of this force means the water is being pulled back into the channel. The force will normally be positive because the water/air interfacial tension is greater than the water/teapot interfacial tension. That's why water droplets on the surface of the teapot tend to spread out instead of rolling up.
So far so good. Now, my model is that if this force is great enough to bring the water stream to a stop the water will dribble down the edge, while if the force smaller than this the water will flow cleanly off. This is obviously an approximation because it's possible that the bottom of the water stream may slow and dribble while to top flows cleanly, but let's go with this and see where it gets us.
We'll use the fact that force is equal to rate of change of momentum. The momentum of the water flowing off the edge in one second is simply:
$$p = \rho vA \times v$$
because $A = ld$ is the area of the channel, and the velocity $v$ is the length that flows in one second so $vA$ is the volume and therefore $\rho vA$ the mass. So, if the water comes to a stop at the edge the rate of change of momentum is $p$, and therefore we will get dribbling when:
$$ (\gamma_{wa} - \gamma_{wt})\ell > \rho Av^2 $$
or since the velocity is the only thing we can easily vary, we get dribbling when:
$$ v < \sqrt{\frac{(\gamma_{wa} - \gamma_{wt})\ell}{\rho A} }$$
You can now immediately see why the water/teapot interfacial tension matters. Remember $\gamma_{wt}$ is normally less than $\gamma_{wa}$, but if we hydrophobe the teapot surface we make $\gamma_{wt}$ bigger. If we can make it big enough to equal the water/air interfacial tension, so $\gamma_{wa} - \gamma_{wt} = 0$ then our equation becomes:
$$ v < \sqrt{\frac{zero \times\ell}{\rho A} }$$
so we will never get dribbling.
Note also that the dribbling velocity depends on the ratio $\ell/A$ i.e. the ratio of the channel perimeter to it's area. This means a semi-circular channel would be less likely to dribble than a wide shallow channel.
Best Answer
The material from which the teabag is made is porous and allows air to pass through it. So if you gently lower a teabag into liquid the rising liquid expels the air through the (dry) upper part of the teabag and the teabag sinks.
However, if you wet the material of the teabag then the water spreads out over the material to form a liquid film that seals all the pores in the material, so the wet material will not allow air to pass through it. If you managed to uniformly wet the material of teabag before dropping it into the tea then you have in effect created a bubble of air and the teabag will float.
These two scenarios are extremes, and in practice the wetting is rather more chaotic. Whether any air is trapped in the teabag will depend on how exactly the water pours over it, and I'm not sure I care to make any predictions without trying the experiment (and since I'm a coffee drinker not a tea drinker I have no tea bags to hand).