Hwlau is correct about the book but the answer actually isn't that long so I think I can try to mention some basic points.
Path integral
One approach to quantum theory called path integral tells you that you have to sum probability amplitudes (I'll assume that you have at least some idea of what probability amplitude is; QED can't really be explained without this minimal level of knowledge)) over all possible paths that the particle can take.
Now for photons probability amplitude of a given path is $\exp(i K L)$ where $K$ is some constant and $L$ is a length of the path (note that this is very simplified picture but I don't want to get too technical so this is fine for now). The basic point is that you can imagine that amplitude as a unit vector in the complex plane. So when doing a path integral you are adding lots of short arrows (this terminology is of course due to Feynman). In general for any given trajectory I can find many shorter and longer paths so this will give us a nonconstructive interference (you will be adding lots of arrows that point in random directions). But there can exist some special paths which are either longest or shortest (in other words, extremal) and these will give you constructive interference. This is called Fermat's principle.
Fermat's principle
So much for the preparation and now to answer your question. We will proceed in two steps. First we will give classical answer using Fermat's principle and then we will need to address other issues that will arise.
Let's illustrate this first on a problem of light traveling between points $A$ and $B$ in free space. You can find lots of paths between them but if it won't be the shortest one it won't actually contribute to the path integral for the reasons given above. The only one that will is the shortest one so this recovers the fact that light travels in straight lines. The same answer can be recovered for reflection. For refraction you will have to take into account that the constant $K$ mentioned above depends on the index of refraction (at least classically; we will explain how it arises from microscopic principles later). But again you can arrive at Snell's law using just Fermat's principle.
QED
Now to address actual microscopic questions.
First, index of refraction arises because light travels slower in materials.
And what about reflection? Well, we are actually getting to the roots of the QED so it's about time we introduced interactions. Amazingly, there is actually only one interaction: electron absorbs photon. This interaction again gets a probability amplitude and you have to take this into account when computing the path integral. So let's see what we can say about a photon that goes from $A$ then hits a mirror and then goes to $B$.
We already know that the photon travels in straight lines both between $A$ and the mirror and between mirror and $B$. What can happen in between? Well, the complete picture is of course complicated: photon can get absorbed by an electron then it will be re-emitted (note that even if we are talking about the photon here, the emitted photon is actually distinct from the original one; but it doesn't matter that much) then it can travel for some time inside the material get absorbed by another electron, re-emitted again and finally fly back to $B$.
To make the picture simpler we will just consider the case that the material is a 100% real mirror (if it were e.g. glass you would actually get multiple reflections from all of the layers inside the material, most of which would destructively interfere and you'd be left with reflections from front and back surface of the glass; obviously, I would have to make this already long answer twice longer :-)). For mirrors there is only one major contribution and that is that the photon gets scattered (absorbed and re-emitted) directly on the surface layer of electrons of the mirror and then flies back.
Quiz question: and what about process that the photon flies to the mirror and then changes its mind and flies back to $B$ without interacting with any electrons; this is surely a possible trajectory we have to take into account. Is this an important contribution to the path integral or not?
Looks like you are already familiar with the classical explanation but are still curious about the quantum version of it.
2.phase difference between absorbed and emitted light
Yeah, this is essentially the lowest order contribution to the phase shift in the photon-electron scattering. Here is the sloppy way to visualize it continuously (this is basically the 'classical EM wave scattering' point of view): you can imagine that the "kinetic energy" (-> frequency) of the "photon" increases as it approaches the atom's potential well and then it goes back to its normal frequency upon leaving the atom. This translates to a net increase in the phase ($(n-1)\omega/c$).
- "drift velocity" of photons ( they aren't the same photons, they are re-emitted all the time)
By "drift velocity" do you mean a pinball-like, zigzag motion of the photon? This won't contribute that much because it requires more scattering (basically it is a higher order process).
And also, I still don't really understand about the detail of the absorption-emission process.
Yes the absorption will still occur in all range of the frequency. The hamiltonian of the atom will be modified by the field (by $- p \cdot E$ where p is the dipole moment of the atom and E is the electric field component of the light). This will give us the required energy level to absorb the photon momentarily, which will be re-emitted again by stimulated+spontaneous emission.
edit: clarification, the term 'energy level' is misleading, since the temporarily 'excited' atom is not in an actual energy eigenstate.
See the diagram here: http://en.wikipedia.org/wiki/Raman_scattering
Best Answer
A good question, and one that confused me for a long time.
In the absense of matter the light wave is massless. However in the presence of polarisable matter the light interacts with the matter and forms a composite system. Now you have a single wavefunction that describes the combined system and you can no longer separate it into a light bit and matter bit. This composite system propagates with a velocity of less than $c$.
The composite system of light and matter is called a polariton although I think strictly speaking the term polariton is reserved for strongly interacting systems such as Bose-Einstein condensates, where the speed of light can be reduced to only a few metres per second.