Probability Integral Transform – Proving Without Assuming Strictly Increasing CDF

cumulative distribution functionprobability

I know that the proof of the probability integral transform has been given multiple times on this site. However, the proofs I found use the hypothesis that the CDF $F_X(x)$ is strictly increasing (together, of course, with the hypothesis that $X$ is a continuous random variable). I know that actually the only required hypothesis is that $X$ is a continuous random variable, and strict monotonicity is not required. Can you show me how?

Since I'm already here, I also take the occasion to ask for a simple application of the probability integral transform 🙂 can you show me that, if $X$ has CDF $F_X(x)$ and $Y$ is the truncation of $X$ to $[a,b]$, then $Y$ is distributed as $F_X^{-1}(U)$ where $U\sim[F_X(a),F_X(b)]$?

Best Answer

In the wikipedia link provided by the OP, the probability integral transform in the univariate case is given as follows

Suppose that a random variable $X$ has a continuous distribution for which the cumulative distribution function(CDF) is $F_X$. Then the random variable $Y=F_X(X)$ has a uniform distribution.
PROOF
Given any random variable $X$, define $Y = F_X (X)$. Then:

$$ \begin{align} F_Y (y) &= \operatorname{Prob}(Y\leq y) \\ &= \operatorname{Prob}(F_X (X)\leq y) \\ &= \operatorname{Prob}(X\leq F^{-1}_X (y)) \\ &= F_X (F^{-1}_X (y)) \\ &= y \end{align} $$

$F_Y$ is just the CDF of a $\mathrm{Uniform}(0,1)$ random variable. Thus, $Y$ has a uniform distribution on the interval $[0, 1]$.

The problem with the above is that it is not made clear what the symbol $F_X^{-1}$ represents. If it represented the "usual" inverse (that exists only for bijections), then the above proof would hold only for continuous and strictly increasing CDFs. But this is not the case, since for any CDF we work with the quantile function (which is essentially a generalized inverse),

$$F_Z^{-1}(t) \equiv \inf \{z : F_Z(z) \geq t \}, \;\;t\in (0,1)$$

Under this definition the wikipedia series of equalities continue to hold, for continuous CDFs. The critical equality is

$$\operatorname{Prob}(X\leq F^{-1}_{X} (y)) = \operatorname{Prob}(X\leq \inf \{x : F_X(x) \geq y \})= \operatorname{Prob}(F_X (X)\leq y)$$

which holds because we are examining a continuous CDF. This in practice means that its graph is continuous (and without vertical parts, since it is a function and not a correspondence). In turn, these imply that the infimum (the value of inf{...}), denote it $x(y)$, will always be such that $F_X(x(y)) = y$. The rest is immediate.

Regarding CDFs of discrete (or mixed) distributions, it is not (cannot be) true that $Y=F_X(X)$ follows a uniform $U(0,1)$, but it is still true that the random variable $Z=F_{X}^{-1}(U)$ has distribution function $F_X$ (so the inverse transform sampling can still be used). A proof can be found in Shorack, G. R. (2000). Probability for statisticians. ch.7.

Related Question