In classical mechanics you construct an action (involving a Lagrangian in arbitrary generalized coordinates, a Hamiltonian in canonical coordinates [to make your EOM more "convenient & symmetric"]), then extremizing it gives the equations of motion. Alternatively one can find a first order PDE for the action as a function of it's endpoints to obtain the Hamilton-Jacobi equation, & the Poisson bracket formulation is merely a means of changing variables in your PDE so as to ensure your new variables are still characteristics of the H-J PDE (i.e. solutions of the EOM – see No. 37). All that makes sense to me, we're extremizing a functional to get the EOM or solving a PDE which implicitly assumes we've already got the solution (path of the particle) inside of the action that leads to the PDE. However in quantum mechanics, at least in the canonical quantization I think, you apparently just take the Hamiltonian (the Lagrangian in canonical coordinates) & mish-mash this with ideas from changing variables in the Hamilton-Jacobi equation representation of your problem so that you ensure the coordinates are characteristics of your Hamilton-Jacobi equation (i.e. the solutions of the EOM), then you put these ideas in some new space for some reason (Hilbert space) & have a theory of QM. Based on what I've written you are literally doing the exact same thing you do in classical mechanics in the beginning, you're sneaking in classical ideas & for some reason you make things into an algebra – I don't see why this is necessary, or why you can't do exactly what you do in classical mechanics??? Furthermore I think my questions have some merit when you note that Schrodinger's original derivation involved an action functional using the Hamilton-Jacobi equation. Again we see Schrodinger doing a similar thing to the modern idea's, here he's mish-mashing the Hamilton-Jacobi equation with extremizing an action functional instead of just extremizing the original Lagrangian or Hamiltonian, analogous to modern QM mish-mashing the Hamiltonian with changes of variables in the H-J PDE (via Poisson brackets).
What's going on in this big Jigsaw? Why do we need to start mixing up all our pieces, why can't we just copy classical mechanics exactly – we are on some level anyway, as far as I can see… I can understand doing these things if they are just convenient tricks, the way you could say that invoking the H-J PDE is just a trick for dealing with Lagrangians & Hamiltonians, but I'm pretty sure the claim is that the process of quantization simply must be done, one step is just absolutely necessary, you simply cannot follow the classical ideas, even though from what I've said we basically are just doing the classical thing – in a roundabout way. It probably has something to do with complex numbers, at least partially, as mentioned in the note on page 276 here, but I have no idea as to how to see that & Schrodinger's original derivation didn't assume them so I'm confused about this.
To make my questions about quantization explicit if they aren't apparent from what I've written above:
a) Why does one need to make an algebra out of mixing the Hamiltonian with Poisson brackets?
(Where this question stresses the interpretation of Hamiltonian's as Lagrangian's just with different coordinates, & Poisson brackets as conditions on changing variables in the Hamilton-Jacobi equation, so that we make the relationship to CM explicit)
b) Why can't quantum mechanics just be modelled by extremizing a Lagrangian, or solving a H-J PDE?
(From my explanation above it seems quantization smuggles these idea's into it's formalism anyway, just mish-mashing them together in some vector space)
c) How do complex numbers relate to this process?
(Are they the reason quantum mechanics radically differs from classical mechanics. If so, how does this fall out of the procedure as inevitable?)
Apologies if these weren't clear from what I've written, but I feel what I've written is absolutely essential to my question.
Edit: Parts b) & c) have been nicely answered, thus part a) is all that remains, & it's solution seems to lie in this article, which derives the time dependent Schrodinger equation (TDSE) from the TISE. In other words, the TISE is apparently derived from classical mechanical principles, as Schrodinger did it, then at some point in the complicated derivation from page 12 on the authors reach a point at which quantum mechanical assumptions become absolutely necessary, & apparently this is the reason one assumes tons of axioms & feels comfortable constructing Hilbert spaces etc… Thus elucidating how this derivation incontravertibly results in quantum mechanical assumptions should justify why quantization is necessary, but I cannot figure this out from my poorly-understood reading of the derivation. Understanding this is the key to QM apparently, unless I'm mistaken (highly probable) thus if anyone can provide an answer in light of this articles contents that would be fantastic, thank you!
Best Answer
Concerning point c), on how complex numbers come into quantum theory:
This has a beautiful conceptual explanation, I think, by applying Lie theory to classical mechanics. The following is taken from what I have written on the nLab at quantization -- Motivation from classical mechanics and Lie theory. See there for more pointers and details:
Quantization of course was and is motivated by experiment, hence by observation of the observable universe: it just so happens that quantum mechanics and quantum field theory correctly account for experimental observations where classical mechanics and classical field theory gives no answer or incorrect answers. A historically important example is the phenomenon called the “ultraviolet catastrophe”, a paradox predicted by classical statistical mechanics which is not observed in nature, and which is corrected by quantum mechanics.
But one may also ask, independently of experimental input, if there are good formal mathematical reasons and motivations to pass from classical mechanics to quantum mechanics. Could one have been led to quantum mechanics by just pondering the mathematical formalism of classical mechanics? (Hence more precisely: is there a natural Synthetic Quantum Field Theory?)
The following spells out an argument to this effect. It will work for readers with a background in modern mathematics, notably in Lie theory, and with an understanding of the formalization of classical/prequantum mechanics in terms of symplectic geometry.
So to briefly recall, a system of classical mechanics/prequantum mechanics is a phase space, formalized as a symplectic manifold (X,ω). A symplectic manifold is in particular a Poisson manifold, which means that the algebra of functions on phase space X, hence the algebra of classical observables, is canonically equipped with a compatible Lie bracket: the Poisson bracket. This Lie bracket is what controls dynamics in classical mechanics. For instance if H∈C ∞(X) is the function on phase space which is interpreted as assigning to each configuration of the system its energy – the Hamiltonian function – then the Poisson bracket with H yields the infinitesimal time evolution of the system: the differential equation famous as Hamilton's equations.
Something to take notice of here is the infinitesimal nature of the Poisson bracket. Generally, whenever one has a Lie algebra 𝔤, then it is to be regarded as the infinitesimal approximation to a globally defined object, the corresponding Lie group (or generally smooth group) G. One also says that G is a Lie integration of 𝔤 and that 𝔤 is the Lie differentiation of G.
Therefore a natural question to ask is: Since the observables in classical mechanics form a Lie algebra under Poisson bracket, what then is the corresponding Lie group?
The answer to this is of course “well known” in the literature, in the sense that there are relevant monographs which state the answer. But, maybe surprisingly, the answer to this question is not (at time of this writing) a widely advertized fact that has found its way into the basic educational textbooks. The answer is that this Lie group which integrates the Poisson bracket is the “quantomorphism group”, an object that seamlessly leads to the quantum mechanics of the system.
Before we spell this out in more detail, we need a brief technical aside: of course Lie integration is not quite unique. There may be different global Lie group objects with the same Lie algebra.
The simplest example of this is already one of central importance for the issue of quantization, namely, the Lie integration of the abelian line Lie algebra ℝ. This has essentially two different Lie groups associated with it: the simply connected translation group, which is just ℝ itself again, equipped with its canonical additive abelian group structure, and the discrete quotient of this by the group of integers, which is the circle group U(1)=ℝ/ℤ. Notice that it is the discrete and hence “quantized” nature of the integers that makes the real line become a circle here. This is not entirely a coincidence of terminology, but can be traced back to the heart of what is “quantized” about quantum mechanics.
Namely, one finds that the Poisson bracket Lie algebra 𝔭𝔬𝔦𝔰𝔰(X,ω) of the classical observables on phase space is (for X a connected manifold) a Lie algebra extension of the Lie algebra 𝔥𝔞𝔪(X) of Hamiltonian vector fields on X by the line Lie algebra: ℝ⟶𝔭𝔬𝔦𝔰𝔰(X,ω)⟶𝔥𝔞𝔪(X). This means that under Lie integration the Poisson bracket turns into an central extension of the group of Hamiltonian symplectomorphisms of (X,ω). And either it is the fairly trivial non-compact extension by ℝ, or it is the interesting central extension by the circle group U(1). For this non-trivial Lie integration to exist, (X,ω) needs to satisfy a quantization condition which says that it admits a prequantum line bundle. If so, then this U(1)-central extension of the group Ham(X,ω) of Hamiltonian symplectomorphisms exists and is called… the quantomorphism group QuantMorph(X,ω): U(1)⟶QuantMorph(X,ω)⟶Ham(X,ω). While important, for some reason this group is not very well known, which is striking because it contains a small subgroup which is famous in quantum mechanics: the Heisenberg group.
More precisely, whenever (X,ω) itself has a compatible group structure, notably if (X,ω) is just a symplectic vector space (regarded as a group under addition of vectors), then we may ask for the subgroup of the quantomorphism group which covers the (left) action of phase space (X,ω) on itself. This is the corresponding Heisenberg group Heis(X,ω), which in turn is a U(1)-central extension of the group X itself: U(1)⟶Heis(X,ω)⟶X. At this point it is worth pausing for a second to note how the hallmark of quantum mechanics has appeared as if out of nowhere simply by applying Lie integration to the Lie algebraic structures in classical mechanics:
if we think of Lie integrating ℝ to the interesting circle group U(1) instead of to the uninteresting translation group ℝ, then the name of its canonical basis element 1∈ℝ is canonically ”i”, the imaginary unit. Therefore one often writes the above central extension instead as follows: iℝ⟶𝔭𝔬𝔦𝔰𝔰(X,ω)⟶𝔥𝔞𝔪(X,ω) in order to amplify this. But now consider the simple special case where (X,ω)=(ℝ 2,dp∧dq) is the 2-dimensional symplectic vector space which is for instance the phase space of the particle propagating on the line. Then a canonical set of generators for the corresponding Poisson bracket Lie algebra consists of the linear functions p and q of classical mechanics textbook fame, together with the constant function. Under the above Lie theoretic identification, this constant function is the canonical basis element of iℝ, hence purely Lie theoretically it is to be called ”i”.
With this notation then the Poisson bracket, written in the form that makes its Lie integration manifest, indeed reads [q,p]=i. Since the choice of basis element of iℝ is arbitrary, we may rescale here the i by any non-vanishing real number without changing this statement. If we write ”ℏ” for this element, then the Poisson bracket instead reads [q,p]=iℏ. This is of course the hallmark equation for quantum physics, if we interpret ℏ here indeed as Planck's constant. We see it arises here merely by considering the non-trivial (the interesting, the non-simply connected) Lie integration of the Poisson bracket.
This is only the beginning of the story of quantization, naturally understood and indeed “derived” from applying Lie theory to classical mechanics. From here the story continues. It is called the story of geometric quantization. We close this motivation section here by some brief outlook.
The quantomorphism group which is the non-trivial Lie integration of the Poisson bracket is naturally constructed as follows: given the symplectic form ω, it is natural to ask if it is the curvature 2-form of a U(1)-principal connection ∇ on complex line bundle L over X (this is directly analogous to Dirac charge quantization when instead of a symplectic form on phase space we consider the the field strength 2-form of electromagnetism on spacetime). If so, such a connection (L,∇) is called a prequantum line bundle of the phase space (X,ω). The quantomorphism group is simply the automorphism group of the prequantum line bundle, covering diffeomorphisms of the phase space (the Hamiltonian symplectomorphisms mentioned above).
As such, the quantomorphism group naturally acts on the space of sections of L. Such a section is like a wavefunction, except that it depends on all of phase space, instead of just on the “canonical coordinates”. For purely abstract mathematical reasons (which we won’t discuss here, but see at motivic quantization for more) it is indeed natural to choose a “polarization” of phase space into canonical coordinates and canonical momenta and consider only those sections of the prequantum line bundle which depend only on the former. These are the actual wavefunctions of quantum mechanics, hence the quantum states. And the subgroup of the quantomorphism group which preserves these polarized sections is the group of exponentiated quantum observables. For instance in the simple case mentioned before where (X,ω) is the 2-dimensional symplectic vector space, this is the Heisenberg group with its famous action by multiplication and differentiation operators on the space of complex-valued functions on the real line.