This is a question I've been asked several times by students and I tend to have a hard time phrasing it in terms they can understand. This is a natural question to ask and it is not usually well covered in textbooks, so I would like to know of various perspectives and explanations that I can use when teaching.
The question comes up naturally in what is usually students' second course in quantum physics / quantum mechanics. At that stage one is fairly comfortable with the concept of wavefunctions and with the Schrödinger equation, and has had some limited exposure to operators. One common case, for example, is to explain that some operators commute and that this means the corresponding observables are 'compatible' and that there exists a mutual eigenbasis; the commutation relation is usually expressed as $[A,B]=0$ but no more is said about that object.
This naturally leaves students wondering
what is, exactly, the physical significance of the object $[A,B]$ itself?
and this is not an easy question. I would like answers to address this directly, ideally at a variety of levels of abstraction and required background. Note also that I'm much more interested in the object $[A,B]$ itself than what the consequences and interpretations are when it is zero, as those are far easier and explored in much more depth in most resources.
One reason this is a hard question (and that commutators are such confusing objects for students) is that they serve a variety of purposes, with only thin connecting threads between them (at least as seen from the bottom-up perspective).
-
Commutation relations are usually expressed in the form $[A,B]=0$ even though, a priori, there appears to be little motivation for the introduction of such terminology.
-
A lot of stock is placed behind the canonical commutation relation $[x,p]=i \hbar$, though it is not always clear what it means.
(In my view, the fundamental principle that this encodes is essentially de Broglie's relation $\lambda=h/p$; this is made rigorous by the Stone-von Neumann uniqueness theorem but that's quite a bit to expect a student to grasp at a first go.)
-
From this there is a natural extension to the Heisenberg Uncertainty Principle, which in its general form includes a commutator (and an anticommutator, to make things worse). Canonically-conjugate pairs of observables are often introduced, and this is often aided by observations on commutators. (On the other hand, the energy-time and angle-angular momentum conjugacy relations cannot be expressed in terms of commutators, making things even fuzzier.)
-
Commutators are used very frequently, for example, when studying the angular momentum algebra of quantum mechanics. It is clear they play a big role in encoding symmetries in quantum mechanics but it is hardly made clear how and why, and particularly why the combination $AB-BA$ should be important for symmetry considerations.
This becomes even more important in more rigorous treatments of quantum mechanics, where the specifics of the Hilbert space become less important and the algebra of observable operators takes centre stage. The commutator is the central operation of that algebra, but again it's not very clear why that combination should be special.
-
An analogy is occasionally made to the Poisson brackets of hamiltonian mechanics, but this hardly helps – Poisson brackets are equally mysterious. This also ties the commutator in with time evolution, both on the classical side and via the Heisenberg equation of motion.
I can't think of any more at the moment but they are a huge number of opposing directions which can make everything very confusing, and there is rarely a uniting thread. So: what are commutators, exactly, and why are they so important?
Best Answer
Self adjoint operators enter QM, described in complex Hilbert spaces, through two logically distinct ways. This leads to a corresponding pair of meanings of the commutator.
The former way is in common with the two other possible Hilbert space formulations (real and quaternionic one): Self-adjoint operators describe observables.
Two observables can be compatible or incompatible, in the sense that they can or cannot be measured simultaneously (corresponding measurements disturb each other when looking at the outcomes). Up to some mathematical technicalities, the commutator is a measure of incompatibility, in view of the generalizations of Heisenberg principle you mention in your question. Roughly speaking, the more the commutator is different form $0$, the more the observables are mutually incompatible. (Think of inequalities like $\Delta A_\psi \Delta B_\psi \geq \frac{1}{2} |\langle \psi | [A,B] \psi\rangle|$. It prevents the existence of a common eigenvector $\psi$ of $A$ and $B$ - the observables are simultaneously defined - since such an eigenvector would verify $\Delta A_\psi =\Delta B_\psi =0$.)
The other way self-adjoint operators enter the formalism of QM (here real and quaternionic versions differ from the complex case) regards the mathematical description of continuous symmetries. In fact, they appear to be generators of unitary groups representing (strongly continuous) physical transformations of the physical system. Such a continuous transformation is represented by a unitary one-parameter group $\mathbb R \ni a \mapsto U_a$. A celebrated theorem by Stone indeed establishes that $U_a = e^{iaA}$ for a unique self-adjoint operator $A$ and all reals $a$. This approach to describe continuous transformations leads to the quantum version of Noether theorem just in view of the (distinct!) fact that $A$ also is an observable.
The action of a symmetry group $U_a$ on an observable $B$ is made explicit by the well-known formula in Heisenberg picture:
$$B_a := U^\dagger_a B U_a$$
For instance, if $U_a$ describes rotations of the angle $a$ around the $z$ axis, $B_a$ is the analog of the observable $B$ measured with physical instruments rotated of $a$ around $z$.
The commutator here is a first-order evaluation of the action of the transformation on the observable $B$, since (again up to mathematical subtleties especially regarding domains):
$$B_a = B -ia [A,B] +O(a^2) \:.$$
Usually, information encompassed in commutation relations is very deep. When dealing with Lie groups of symmetries, it permits to reconstruct the whole representation (there is a wonderful theory by Nelson on this fundamental topic) under some quite mild mathematical hypotheses. Therefore commutators play a crucial role in the analysis of symmetries.