Background
Let me start this question by a long introduction, because I assume that only few readers will be familiar with the theory of partial coherent light and concepts like a mutual coherence function or a mutual intensity. The coherency matrix and Stokes parameters descriptions of partially polarized light are related concepts which are more widely known.
Correct treatment of partial coherent light is important for an appropriate modeling of optical pattern transfer in computer simulations of proximity and projection lithography as currently used by the semiconductor manufacturing industry. When I came to this industry, my previous optics "training" was insufficient in this area. I found chapter X "Interference and diffraction with partially coherent light" in Principles of Optics by Max Born and Emil Wolf most helpful for filling my gaps in this area. Later, I also "browsed" through "Statistical Optics" by Joseph W. Goodman, which has a nice paragraph in the introduction explaining why insufficient familiarity with statistical optics is so common:
Surely the preferred way to solve a problem must be the deterministic way, with statistics entering only as a sign of our own weakness or limitations. Partially as a consequence of this viewpoint, the subject of statistical optics is usually left for the more advanced students, particularly those with a mathematical flair.
The interesting thing is that Hermitian matrices and eigenvalue decompositions like the Karhunen-Loève expansion are used quite routinely in this field, and they somehow feel quite similar to modeling of coherence and decoherence in quantum-mechanics. I know that there are important obvious (physical) difference between the two fields, but my actual question is what they have in common.
Question
Some elementary experiments like the double slit experiment are often used to illustrate the particle wave duality of light. However, the theory of partially coherent light is completely sufficient to describe and predict the outcome of these experiments. There are no particles at all in the theory of partially coherent light, only waves, statistics and uncertainty. The global phase is an unobservable parameter in both theories, but the amplitude of a wave function is only important for the theory of partial coherent light and is commonly normalized away in quantum-mechanics. This leads to a crucial difference with respect to the possible transformations treated by the respective theories. But is this really a fundamental difference, or just a difference in the common practices of the respective theories? How much of the strange phenomena of quantum-mechanics can be explained by the theory of partial coherent light alone, without any reference to particles or measurement processes?
More information on what I would actually like to learn
One reason for this question is to find out how much familiarity with partial coherence can be assumed when asking questions here. Therefore it explains why this familiarity cannot be taken for granted, and is written in a style to allow quite general answers. However, it also contains specific questions, indicated by question marks:
- How is the theory of partial coherent light related to quantum-mechanics?
- … the amplitude of a wave function … But is this really a fundamental difference, or just a difference in the common practices of the respective theories?
- How much of the strange phenomena of quantum-mechanics can be explained by the theory of partial coherent light alone, without any reference to particles or measurement processes?
Don't be distracted by my remark about the double slit experiment. Using it to illustrate the particle wave duality of light seemed kind of cheating to me long before I had to cope with partial coherence. I could effortlessly predict the outcome of all these supposedly counter-intuitive experiments without even being familiar with the formalism of quantum-mechanics. Still, the outcome of these experiments is predicted correctly by quantum-mechanics, and independently by the theory of partial coherent light. So these two theories do share some common parts.
An interesting aspect of the theory of partial coherent light is that things like the mutual intensity or the Stokes parameters can in principle be observed. A simple analogy to the density matrix in quantum-mechanics is the coherency matrix description of is partial polarization. It can be computed in terms of the Stokes parameters
$$J=\begin{bmatrix}
E(u_{x}u_{x}^{\ast})&E(u_{x}u_{y}^{\ast})\\
E(u_{y}u_{x}^{\ast})&E(u_{y}u_{y}^{\ast})
\end{bmatrix}=\frac12\begin{bmatrix}
S_0+S_1&S_2+iS_3\\
S_2-iS_3&S_0-S_1
\end{bmatrix}
$$
and hence can in principle be observed. But can the density matrix in quantum-mechanics in principle be observed? Well, the measurement process of the Stokes parameters can be described by the following Hermitian matrices $\hat{S}_0=\begin{bmatrix}1&0\\0&1\end{bmatrix}$, $\hat{S}_1=\begin{bmatrix}1&0\\0&-1\end{bmatrix}$, $\hat{S}_2=\begin{bmatrix}0&1\\1&0\end{bmatrix}$ and
$\hat{S}_3=\begin{bmatrix}0&i\\-i&0\end{bmatrix}$. Only $\hat{S}_0$ commutes with all other Hermitian matrices, which somehow means that each individual part of the density matrix can be observed in isolation, but the entire density matrix itself is not observable. But we don't measure all Stokes parameters simultaneous either, or at least that's not what we mean when we say that the Stokes parameters can be measured in principle. Also note the relation of the fact that $\hat{S}_0$ commutes with all other Hermitian matrices and the fact that the amplitude of a wave function is commonly normalized away in quantum-mechanics. But the related question is really a serious question for me, because the Mueller calculus for Stokes parameters allows (slightly unintuitive) transformations which seem to be ruled out for quantum-mechanics.
Best Answer
To start with the double slit experiment gives interference even when the beam is composed by one photon at a time. The spot on the screen a photon/particle the statistical accumulation the interference seen as expected classically too.
The joint comes because the photon as a quantum mechanical entity has a wavefunction that is the solutions of Maxwell's equation, treated as operators operating on the wave function. The E=h*nu identity the photon carries is the same as the frequency of the classical solution of Maxwell's equation and that , together with the phase attached to the wavefunction allow the continuity of classical down to quantum single photon level For a formal treatment how classical emerges from an ensemble of photons have a look at this blog entry.
There exists a fundamental difference between classical and quantum equations and their solutions, but also a continuity at the interface.
Edit after question edit
This needs somebody familiar with the formalism of both, but I believe the connection should follow the method in the link, how classical electromagnetic beams emerge from an ensemble of photons.
]>... the amplitude of a wave function ... But is this really a fundamental difference, or just a difference in the common practices of the respective theories?
the square of the wavefunction is the connection with predictions and experiments in quantum mechanics, it is the probabilistic nature that makes the difference with the classical framework, as far as I know.
Phenomenon is " an observable" , observing something implies a measurement process, measurement implies interaction, picking up a point that will contribute to the quantum mechanical probability distribution (or building up the distribution itself by continuous observations) so there is an inherent contradiction in this part of the question.