[Math] The meaning of the connection between power spectral density and auto correlation

fourier analysisintuitionstochastic-processes

I know that if we have a signal $x(t)$, then its Fourier transform would be the signal in the frequency space, which I understand to be how much of each frequency exists in the x(t) signal.
$ \mathcal{F}\{x(t)\} = X(f)$

Now I'm studying stochastic processes, and there seems to be something a little similar here: the Fourier transform of the auto correlation (of a stochastic process $Y(t)$) is the power spectral density:

$\mathcal{F}\{R_Y(\tau)\} = S_Y(f)$.

My question is about the intuitive understanding of this.

Why is the PSD a Fourier transform of the auto correlation?

What is the connection here between the meaning of the auto correlation and the meaning of the power spectral density, a connection which causes them to be related by a Fourier transform, such as $x(t)\rightarrow X(f)$?

Best Answer

It may be helpful to look at the more familiar deterministic case first. In order to obtain an analogy to the stochastic case you shouldn't use the relation $x(t)\Longleftrightarrow X(f)$, but you have to consider the energy spectral density $|X(f)|^2$. Its inverse Fourier transform is the deterministic autocorrelation function $\rho_x(\tau)$:

$$|X(f)|^2=X(f)X^*(f)\Longleftrightarrow x(t)*x^*(-t)=\int_{-\infty}^{\infty}x^*(t)x(t+\tau)dt=\rho_x(\tau)\tag{1}$$

Clearly the multiplication necessary to obtain a quadratic quantity in the frequency domain corresponds to a convolution in the time domain, which is equivalent to a correlation.

The relation in the stochastic case is analogous to the relation (1). The power spectrum of a wide-sense stationary process is defined by

$$S_X(f)=\lim_{T\rightarrow\infty}\frac{1}{2T}E\left\{\left|\int_{-T}^{T}X(t)e^{-2\pi ift}dt\right|^2\right\}\tag{2}$$

where $E\{\cdot\}$ denotes the expectation operator. It can be shown that under certain conditions (cf. Einstein-Wiener-Khinchin theorem) (2) is the Fourier transform of the stochastic autocorrelation function $R_X(\tau)=E\{X^*(t)X(t+\tau)\}$.

In both cases you see the fact that a quadratic quantity in the frequency domain (energy spectral density in the deterministic case, power spectral density in the stochastic case) corresponds to a correlation - which is essentially the same as a convolution - in the time domain. This is the main difference to the simple (non-quadratic) relation $x(t)\Longleftrightarrow X(f)$ in the deterministic case.