Distributions – How to Graphically Transform One PDF into Another

distributionslognormal distributionnormal distribution

To understand what I mean, let's use two well-known distributions: the normal and lognormal ones.

From the dataset point of view, if you take normally-distributed data and take their exponential, you get lognormally-distributed data (norm -exp-> lognorm).

So the "idea" was: why not using the PDF directy and transform the normal PDF to the lognormal one by taking the exponential of its x-axis. The reasoning was that this x-axis represents my data: taking the exponential of these data or the exponential of the x-axis of the corresponding PDF should do the same.

But I did not even bother to do it: mathematically, this is not how one can transform the normal PDF to the lognormal one!

What I don't understand is why just taking the exponential of the x-axis of the PDF does not work. I mean, not mathematically (I get that there is a $1/x$ in the lognormal PDF!), but what is conceptually wrong with "just stretching" the axis?

Best Answer

You're heading in the right direction with your thoughts on considering the cdf.

Consider some random variable, $X$ with cdf $F_X(x)$ and density $f_X(x)$. To make things simple, consider applying some monotonic increasing transformation, $t$ on $X$, giving $Y=t(X)$. The new variable $Y$ has cdf $F_Y(y)$ and density $f_Y(y)$. Then:

$F_Y(y) = P(Y\leq y) = P(t(X)\leq y) = P(X\leq t^{-1}(y)) = F_X(t^{-1}(y))$

(By plotting $F_X(t^{-1}(y))$ against $y$ , this has the "stretching" effect on the x-axis you mentioned - the values on the vertical axis are unchanged but are shifted on the horizontal axis.)

Now we can see where that $\frac{1}{x}$ term came from in the lognormal pdf.
Recall we had:

$F_Y(y) = F_X(t^{-1}(y))$

So

$f_Y(y) = \frac{d}{dy} F_X(t^{-1}(y)) = f_X(t^{-1}(y))\cdot \frac{d}{dy}t^{-1}(y)$

A similar result can be derived for monotonic decreasing transformations, yielding the more general result for invertible transformations:

$f_Y(y) = \frac{d}{dy} F_X(t^{-1}(y)) = f_X(t^{-1}(y))\cdot |\frac{d}{dy}t^{-1}(y)|$

When $t$ is the $\exp$ function, $t^{-1}$ is the log, which has the reciprocal as its derivative.

So you do that axis transformation you thought about, but you then have an additional factor, the Jacobian of the transformation, which changes the height. So far it's quite clear that we must have that term when we go to the pdf from the CDF.

But we can also explain more directly why we need it:

Loosely, note that if you have a very small interval $[x,x+\delta x)$ for which $f$ is effectively constant (so the area is effectively $f(x)\,\delta x$), if you stretch the axis by transforming it as for the cdf, the total area in the transformed small interval is changed by the stretching, but the probability of being in the interval is unchanged. So to preserve the probability represented by the small area, you need to "undo" the impact of the stretching on the small area so that it still represents the probability. The area is kept the same by modifying the height. (This is what the Jacobian does -- preserve small areas.)

![enter image description here

Note that dividing in our example by $t'(x)=\exp(x)$ is in that case the same as dividing by $y$, which is the scaling factor we get from the Jacobian calculation above for $t(x)=\exp(x)$.

Related Question