Solved – Intuition for how the cumulative probability distribution can be derived from probability density function

distributionsintuitionprobability

If we have a probability density function, we can integrate it and now we have a cumulative probability distribution function. Can you help my intuition on why this works?

Best Answer

Loosely speaking...

Integration can be loosely thought as an analog to summation. Imagine that you had a discrete random variable following the categorical distribution, with $\Pr(X = x_i) = p_i$ for $i = 1,\dots,n$ and $\sum_i p_i = 1$. In such case $\Pr(X\le1) = p_1$, what is pretty obvious, and $\Pr(X\le2) = p_1 +p_2$, in more general terms

$$ \Pr(X \le k) = \sum_{j=1}^k p_j $$

so

$$ F(k) = \sum_{j=1}^k f(j) $$

Now recall that in continuous random variables we have infinitely many $x$'s, so $\Pr(X=x) = 0$ and because of that we use probability density functions, that tell us about "probabilities per foot". Imagine that $X$ is a continuous random variable. Imagine that you bin the values of $X$ in the $[x_i, x_i+\Delta x]$ bins, now given the axioms of probability it follows that probabilities of all the intervals need to sum to unity

$$ \sum_i \Pr([x_i, x_i+\Delta x]) = 1 $$

this can be re-refined in terms of probability densities (probabilities per unit), as described by Kruschke (2015), as

$$ \sum_i \frac{\Pr([x_i, x_i+\Delta x])}{\Delta x} \Delta x = 1 $$

now as $\Delta x \to 0$ this becomes an integral

$$ \int f(x) \, d x = 1 $$

So in continuous case we use integration to "sum" the probability densities up to a given value

$$ \Pr(X \le x) = F(x) = \int_{-\infty}^x f(t) \, dt $$

To learn more you could check the Wikipedia page on integrals, Khan academy videos on Riemann approximation of integrals and chapter 4 of the book by Kruschke

Kruschke, J.K. (2015). Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan. Elsevier.

since all of them pay pretty much attention to providing an intuitive explanation of integration.