Here is an analogy that I like to use: (even though it is not really a correct physical explanation)
Imagine that you are out riding your segway over some strange surfaces, that each have a number $n_i$ that controls the speed that a segway wheel travels over it according to the formula $v_i=v_0/n_i$. Now imagine that you cross a straight boundary between two surfaces at an angle. Because of the angle, one wheel will cross the boundary before the other. If $n_i$ is higher for the entered surface this wheel will go slower than the other until it too crosses the boundary, which will cause the segway to turn towards the normal of the boundary. Similarly, if $n_i$ is lower for the entered surface, the first wheel to enter will go faster, and the segway will turn from the normal.
If you do the calculations for the segway you will get the the same results as for the wavefront explanation (basically Snell's law), but I really like how this analogy works with your intuition.
It seems your question comes down to "Why does light at least somewhat follow the curvature of the earth?".
The answer is indeed refraction. Light has different speeds in different transparent substances, always slower than in vacuum. From this differing speed, you can show that a light beam is bent at the boundary between substances with different index of refraction, which is the ratio of how much light slows down in the substance compared to vacuum. Camera lenses, eyeglasses, etc, harness this principle deliberately.
The speed of light in air is close to that in vacuum, but not exactly the same. Put another way, the index of refraction of air is almost 1, but not quite. Furthermore this index of refraction varies with the density of the air. To convince yourself of this, imagine the limiting case where you measure index of refraction of air as the pressure is gradually lowered. When it gets to 0, the index of refraction must be 1 by definition. The index of refraction of air therefore varies smoothly as a function of pressure.
Now think of the air envelope around the earth. Obviously there is a pressure gradient with altitude. When you get high enough, the atmosphere is gone and you have only the vacuum (almost) of space. In this case there isn't a sharp boundary like there is when light enters a glass lens. However, the gradient still bends light, in this case smoothly over some distance, as apposed to abruptly at the air/glass boundary in the lens example. This vertical pressure gradient, and therefore index of refraction gradient, causes light to bend a little when shot horizontally thru the atmosphere.
However, there is more to it than this general effect. The atmosphere is not uniform at any one altitude. As you know, there is wind, pockets of hot and cold air, rising thermals, cold downdrafts, and lots of phenomena that are much more significant locally than the general decrease in pressure vertically. The air can have different layers at different temperatures, and the interface between layers can be much more abrupt than the general trend of decreased pressure with altitude.
Shooting a light beam with the right atmospheric conditions can exhibit much more bending than in the general average case. A mirage is a good example of this. Light from the horizon is refracted by the relatively sharp boundary at the top of a thin hot layer of air warmed by the ground. From far enough away to that the light is at a very glancing angle, you "see" sky light reflected off of what looks like the ground. This gives the visual impression of a lake, since a lake would similarly reflect sky light in normal cases even when there are no special atmospheric effects.
In the case of a mirage, light is actually bent upward. Light can just as well be bent downward using similar boundaries of layers in the atmosphere. It depends on the position of the emitter and receiver relative to the index of refraction gradients in the atmosphere.
If you were to carry out these experiments on the moon, which has no practical atmosphere for this purpose, light beams would indeed to "straight". You won't see mirages on the moon, for example.
Best Answer
Yes, this a pretty and very simple experiment. And the explanation likewise.
It is indeed about refractive index. The glass seen from above (empty to the left):
The refractive index $n$ (or index of refraction) is a material proporty. It is the relation between the speed $v$ of light in the material as to that in vacuum $c$:
$$n=\frac{c}{v}$$
Each material has it's own value, $n_{air}$, $n_{glass}$, $n_{water}$.
For air$\to$glass the light bends. For glass$\to$air on the inside, it bends back. The ray is shifted a bit to the side but still towards you. Same but opposite situation when it hits the other inside of the glass. The ray is then back where it started.
With water in the glass, for air$\to$glass the light still bends inwards. For glass$\to$water it bends even more. It will reach the other side of the glass. For water$\to$glass it bends a bit back. For glass$\to$air it bends all the way back. Direction is the same but the ray is shifted to the side.
The change in glass is cancelled out on each side. But the change in water is not.