This is a really tough one:
I would like to understand what it really means to apply the dipole approximation when deriving the selection rules. This question is purely about intuitive understanding because the derivation itself seems way above my level of quantum mechanics.
What I know:
- In classical electrodynamics we can expand our potentials in a multipole expansion. Often, we do not need to consider higher orders of this expansion as they decrease with additional $\frac{1}{r^2}$ factors and thus at reasonable distances won't make much of a difference anymore
- To derive the selection rules, we have to consider the transition dipole moment which looks roughly like this: $M_{if} = \int \psi^*_f \boldsymbol{\mu} \psi_i d\vec{r}$ where $\boldsymbol{\mu}$ is the transition operator
Now, I have looked at a lot of stuff and I have seen you can write this operator down without really taking much benefit. I suppose it somehow contains our potentials and we can write in the form of a multipole expansion so that leaving out all terms is a viable thing.
The confusion: Classically, I would leave out terms if I wanted to know the potential of some charge distribution at a relatively distant point but what is the reasoning of neglecting higher order terms of the multipole expansion in the quantum mechanical picture for let's say absorption or emission? If it is about distances $r$ again then where are these distances?
And ultimately, what does it really mean not to consider for example a magnetic dipole in this case. I have absolutely no intuition for it. I have read that considering those higher order terms, there could be transitions that do not follow the $\Delta l = \pm 1$ rule for example and I wonder wether there is any way to imagine this without going to math right away.
Progress: I know now that the approximation is based on the wavelength of the photon being significantly larger than the extension of an atom and that higher order terms in the expansion scale with this factor $R / \lambda$. This would imply that this approximation should not hold for short wavelength (X-Ray for instance). Unfortunately, I still don't have any explanation for questions above.
Best Answer
It seems to me you have asked a cluster of questions. You asked why electric dipole transition is often the only interaction we are interested in, with all other terms corresponding to higher multipole moments ignored. You also asked (effectively) what is the smallness parameter that justifies this choice. I believe the best way to answer them is to go through the derivation of various terms of contributions to the interaction between an atom and a radiation field. I will omit some details in order to present a clear outline of the argument. My derivations here mostly draws from Cohen-Tannoudji's Quantum Mechanics Vol. 2, Complement A13.
The Hamiltonian of an electron in an electromagnetic field described by vector potential $\mathbf A(\mathbf r, t)$ and scalar potential $q\phi(\mathbf r, t)$ is $$ H = \frac{1}{2m} [\mathbf p - q \mathbf A(\mathbf r, t)]^2 + q\phi(\mathbf r, t)- \frac{q\hbar}{2m} \boldsymbol \sigma \cdot \nabla\times \mathbf A (\mathbf r, t) \>. $$ We can expand it into $$ H(t) = \frac{\mathbf p^2}{2m} + q\phi(\mathbf r, t)- \frac{q}{m}\mathbf p \cdot \mathbf A - \frac{q\hbar}{2m} \boldsymbol \sigma\cdot \nabla\times \mathbf A + \frac{q^2}{2m} \mathbf A^2 \>. $$ The last term can be ignored for ordinary light sources, since the intensity is sufficiently low. Call the third term $W_1$ and the fourth term $W_2$. We can now treat $W_1 +W_2$ as a perturbation to the atomic Hamiltonian (the first and second terms), and analyse it by attempting to expand it with respect to some smallness parameter.
Assuming we are dealing with plane waves polarized in one direction, we can estimate and compare the magnitude of the terms $W_1$ and $W_2$, $$ \frac{W_2}{W_1} \simeq \frac{\frac{q}{m}\hbar |\mathbf k| \mathcal A_0}{\frac{q}{m}p \mathcal A_0} = \frac{\hbar |\mathbf k|}{p} \>, $$ where $\mathcal A_0$ is the amplitude of the vector potential field, $\mathbf k$ its wave vector, and $p$ the momentum of the electron. Since $\hbar/p$ is at at most on the order of the size of the atom which is on the scale of the Bohr radius $a_0$, and $|\mathbf k| = 2\pi/\lambda$, where $\lambda$ typically much greater than the size of the atom, this ratio $W_2/W_1$ is about $a_0 /\lambda$, which is very small. This is a good enough justification for ignoring $W_2$ when we are performing the computation to the zeroth order in $a_0/\lambda$, which is the starting point of many analyses you can find in books or online that are only trying to obtain the electric dipole Hamiltonian. However we are not satisfied with that, so we shall perform a full expansion.
Both $W_1$ and $W_2$ contain an exponential factor $e^{\pm i\mathbf k\cdot \mathbf r}$ as the spatial dependence of $\mathbf A(\mathbf r, t)$. We can then expand it in powers of $\mathbf k\cdot \mathbf r$. Note again that $|\mathbf k| = 2\pi/\lambda$, and $\mathbf r$ is of the order of the size of the atom, so this is of the same order as $W_2/W_1$. Thus if we expand $W_1$ into $W_1^0 + W_2^1 + \dots$ and $W_2$ into $W_2^0 + W_2^1 + \dots$, we will find that to the zeroth order of $\mathbf k \cdot \mathbf r$, we have $W_1^0$, and to the first order we have $W_1^1 + W_2^0$ and so on.
The form of the relevant terms are $W_1^0 = \frac{q}{m} \mathbf p \cdot \partial_t \mathbf E(\mathbf r, t) = - q \mathbf r \cdot \mathbf E(\mathbf r, t)$ (It takes a bit of work to show that these two forms are equivalent, and I'll have Cohen-Tannoudji do that for me.) This is the electric dipole term. To the next order, the magnetic dipole term is $W_2^0 = - \frac{q}{m}(\mathbf L + 2 \mathbf S)\cdot \mathbf B(\mathbf r, t)$, and the electric quadrupole term is $W_1^1 = - \frac{q}{m} \mathbf r\mathbf r\colon \nabla\mathbf E$, where $\colon$ represents the double contraction between the quadrupole moment tensor and the gradient of the electric field. These terms are labeled by the electric and magnetic multipole moments because the respective multipole moment operators appear in them. For the derivation of the multipole moments operators, again refer to Complement E10 of Cohen-Tannoudji.
Now we have illustrated where each of the terms came from, and more importantly how their magnitudes compare. The answer your question as to why electric dipole transitions are prominent is simply that the Hamiltonian for electric dipole transition is much larger in magnitude than the magnetic dipole Hamiltonian, as well as transitions corresponding to higher multipole moments.
As a final note, in your question you described what you knew about multipole expansion in classical electrodynamics, but what you described is how things work in the far field regime, where the frequency is high and we are interested in the radiation far from the source, or $\mathbf k \cdot \mathbf r \gg 1$. In what we are discussing here, we are working in the opposite limit where $\mathbf k \cdot \mathbf r \ll 1$. Though to be more precise, we are not studying radiation from a source but how radiation interacts with the atom, so the situations are not exactly comparable. For more on far and near field approximations and multipole fields, refer to Chapter 9 of Jackson's Electrodynamics.