Let $\phi_{x}(t)= E [ e^{itx}]$ be the characteristic function
If X is a continuous random variable, then:
$\phi_{x}(t)= E [ e^{itx}] = \int e^{itx} f(x)dx$ (being $f(x)$ the probability density function of x)
If X is a discrete random variable, then:
$\phi_{x}(t)= E [ e^{itx}] = \sum_{k} e^{itx_{k}} p(x_{k})$ (being $p(x)$ the probability mass function of x)
To be general Fourier transform can be define as (Ref: https://www.johndcook.com/blog/fourier-theorems/):
$F(s)=\frac{1}{A}\int_{-\infty}^{\infty} e^{iBst}f(t)dt$
The choices that are found in practice are:
$A=\sqrt{2\pi}, B=\pm 1;$
$A=1, B=\pm 2\pi;$
$A=1, B=\pm 1$
Choosing A=1 and B=1 to relate it to the characteristic function, the inverse Fourier transform is:
$f(t)=\frac{1}{2\pi}\int_{-\infty}^{\infty} e^{-ist}F(s)ds$
In continuous case:
$\phi_{x}(t) = \int e^{itx} f(x)dx \rightarrow f(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty} e^{-ixt}\phi_{x}(t)dt$
That is the utility of the characteristic function, it allows me to know the probability function
But in the discrete case I get problems:
If discrete Fourier transform (DFT) is :
$F(s)=\sum_{n=0}^{N-1} e^{-2\pi sn/N}f(n)$
The inverse DFT is:
$f(t)=\frac{1}{N}\sum_{n=0}^{N-1} e^{2\pi sn/N}F(s)$
So, Would the "inverse" characteristic function be
$\phi_{x}(t) = \sum_{k} e^{itx_{k}} p(x_{k}) \rightarrow P(x) = \frac{1}{2\pi N}\sum_{k} e^{-ixt_{k}}\phi_{x}(t_{n})$ ?
So if FT can have different definitions for the continuous case. What happen with the discrete case? How many definitions are?
Best Answer
For the discrete case, you need to look at DTFT, not DFT. N-point DFT assumes that the underlying function is periodic, which is not the case for probability mass functions. You can then apply similar logic for inverse transform:
$$\phi_{x}(t)= E [ e^{itx}] = \sum_{k} e^{itx_{k}} p(x_{k})\rightarrow p(x)=\frac{1}{2\pi}\int_0^{2\pi}e^{-ixt}\phi_x(t)dt$$
For example, characteristic function for Bernouilli RV is $\phi_x(t)=1-p+pe^{it}$. Applying the formula yields the following:
$$\begin{align}p(x)&={1\over 2\pi}\int_0^{2\pi}e^{-ixt} (1-p+pe^{it})dt\\&=(1-p){1\over 2\pi}\underbrace{\int_0^{2\pi} e^{-itx}dt}_{2\pi\delta_x} + p\frac{1}{2\pi}\underbrace{\int_0^{2\pi}e^{-it(x-1)}dt}_{2\pi\delta_{x-1}}\\&=(1-p)\delta_x+p\delta_{x-1}\end{align}$$
where $\delta_x$ is the Kronecker delta function, i.e. it's $1$ if $x=0$, and $0$ otherwise.
This is from Fourier transform perspective. Probability theory has its own principled way of calculating inverses.