Characteristic Functions – Purpose in Probability and Statistics

characteristic functionmathematical-statisticsprobability

I'm hoping that someone can explain, in layman's terms, what a characteristic function is and how it is used in practice. I've read that it is the Fourier transform of the pdf, so I guess I know what it is, but I still don't understand its purpose. If someone could provide an intuitive description of its purpose and perhaps an example of how it is typically used, that would be fantastic!

Just one last note: I have seen the Wikipedia page, but am apparently too dense to understand what is going on. What I'm looking for is an explanation that someone not immersed in the wonders of probability theory, say a computer scientist, could understand.

Best Answer

Back in the day, people used logarithm tables to multiply numbers faster. Why is this? Logarithms convert multiplication to addition, since $\log(ab) = \log(a) + \log(b)$. So in order to multiply two large numbers $a$ and $b$, you found their logarithms, added the logarithms, $z = \log(a) + \log(b)$, and then looked up $\exp(z)$ on another table.

Now, characteristic functions do a similar thing for probability distributions. Suppose $X$ has a distribution $f$ and $Y$ has a distribution $g$, and $X$ and $Y$ are independent. Then the distribution of $X+Y$ is the convolution of $f$ and $g$, $f * g$.

Now the characteristic function is an analogy of the "logarithm table trick" for convolution, since if $\phi_f$ is the characteristic function of $f$, then the following relation holds:

$$ \phi_f \phi_g = \phi_{f * g} $$

Furthermore, also like in the case of logarithms,it is easy to find the inverse of the characteristic function: given $\phi_h$ where $h$ is an unknown density, we can obtain $h$ by the inverse Fourier transform of $\phi_h$.

The characteristic function converts convolution to multiplication for density functions the same way that logarithms convert multiplication into addition for numbers. Both transformations convert a relatively complicated operation into a relatively simple one.

Related Question