I believe your puzzlement comes from confusing two frameworks: the quantum mechanical (photons) and the classical mechanics one, waves.
When one is calculating in terms of classical electromagnetic waves there are classical considerations : refraction, absorption, reflection with their corresponding constants .
When one is zooming in the microcosm and talking of photons, a wave is composed of zillions of photons which go through, each at the velocity of light.
The bulk of the target material is in effect the electric and magnetic fields holding the atoms together to form it: the nuclei are tiny targets and the electrons are small zooming targets. The probability of a single photon to scatter on a nucleus or an electron is miniscule. It interacts/scatters out of its optical ray path with the electric/magnetic fields that are holding the glass or crystal together. The scattering angles are very small in transparent materials thus preserving the optical path, enormous in opaque ones . It is those fields that one has to worry about, not the individual atoms and their excitations.
The photons scatter mostly elastically with the fields holding the solids together with tiny or high cross sections depending on the frequency of light and spacing of the materials. In crystals and glasses the optical frequencies have small probability of interaction.
x rays find most materials transparent because the photons' energy is much larger than the energies available by the fields holding the crystals together, and the scattering angles with the fields are very small, except when they hit the atoms, and then we get x ray crystallography.
Edit after comment
Here is the sequence as I see it:
A classical electromagnetic wave is made up of photons in phase according to the wave description.
There is an enormous number of photons in the wave per second making it up. Here is a useful article which explains how a classical wave is built up from a quantum substrate.
Each photon does not change the atomic or crystalline energy levels going through a transparent material in the quantum mechanical way by emitting a softer photon. It scatters quantum mechanically elastically, through the medium, changing the direction infinitesimally so that it keeps the quantum mechanical phase with its companions and displays transparency. Since a medium has a composite collective electric and magnetic field it is not a simple "electron photon going to electron photon" QED diagram. In the case of a crystal one could have a model of "photon crystal photon crystal" scattering amplitude for example.
The higher the sequential probability of scattering going through a medium the larger the final deflection through it will be, and the higher the over all probability of losing the phase with its companions in the wave.( the thicker the glass the less transparency and image coherence).
The transparency of the medium depends on the ordering of the atoms and molecules composing it so that it allows to keep the coherence between individual photons of the beam. The lower the density the better chance to keep the transparency, viz water and air.
hope this helps conceptually.
Thus is a very common confusion, and it occurs because light is neither a wave nor a particle but instead it's (currently best described as) a quantum field. The wave and particle descriptions are approximations that apply under some circumstances. In particular the photon model is a good way to describe how the electromagnetic field exchanges energy with it's environment. When the light transfers energy to something else the energy transferred is an integral number of photon energies.
So, in your example, glass is clear because for visible light there aren't any energy levels spaced one photon energy apart. Since light can only interact with the glass by exchanging energy in photon sized lumps the interaction can't occur. Glass does absorb in the ultraviolet because photon energy is proportional to the light frequency and at uv frequencies the photon energy is big enough to excite electrons in the glass.
The green pigment in the paint on your wall has been chosen to have electronic excitations that correspond to the photon energy of red and blue light, but none with an energy matching green light. This means that red and blue light falling on the wall is absorbed but green light is reflected. In general with solids, when a light excites an electronic transition and is absorbed, the energy of the excited electrons is dissipated as lattice vibrations. Only in some circumstances is it re-emitted as light, in which case you get fluorescence or phosphorescence. So it isn't the case that light is absorbed and re-emitted as green light. The green light is reflected and stays green, while the other colours are absorbed and their energy ends up heating the wall.
The green light reaching your retina has a photon energy that matches the optical pigments in the M-cones. Thus the light is absorbed (in photon sized chunks) by exciting electrons in the optical pigments.
Best Answer
Very reasonable question. I will try to answer it in an intuitive way.
If you have a scattering medium, photons are reflected in random directions; but when you have a refractive medium, something else happens. The photon is not absorbed and re-emitted: instead, the photon interacts with the electrons in the medium, and since these electrons are somewhat bound to the atoms, a displacement (due to the E field of the photon) results in a restoring force. We can measure the degree of displacement or polarization, and express it in terms of the dielectric constant of the material. If there is a lot of polarization, the dielectric constant is high. And the refractive index can be shown mathematically to scale with the square root of the dielectric constant.
What happens then is that the bound electrons move "behind in phase" with the photon, which results in a phase shift of the EM wave (the original phase of the photon which lost a bit of its amplitude, and a lagging phase of the electron). The photon didn't get destroyed - it got delayed, but maintained its direction.
Now when you have an interface between two materials of different refractive index, and light is incident at an angle to that interface, then a greater phase difference builds up between adjacent beams, which is why light is refracted; but if all the light is traveling through the same medium of constant refractive index, all beams will refract by the same amount and remain parallel.
Which is why you can see through a window.
But if you use ground glass, the surface is no longer flat but has been modified to change the direction - and this results in the image behind the glass becoming fuzzy. The same principle exists in the glass used in many showers (original image from Victoria Elizabeth Barnes's posts on bathroom remodeling:
You can clearly see how the bottom half of the window blurs the image of the trees outside. In the article they call this "pebbled" glass.