In Bishop's Pattern Recognition and Machine Learning, I'm confused by the section on the change of variable of a probability density function. The paragraph concerned is bellow.
So, given a change of variable such as $x=g(y)$, the pdf of $y$ is defined as: $$p_y(y)=p_x(g(y))|g'(y)|$$
How do we get here ? Also, but that may be obvious after an answer to the first question, in equation 1.27, why do we have the absolute value of $\frac{dx}{dy}$ ?
Best Answer
Note that $P(x)=\int_{-\infty}^xp(t)\,\mathrm{d}t$ is the probability that the variable will be less than $x$. Thus, if $g$ is a monotonically increasing function, $P(g(x))$ is the probability that the variable will be less than $g(x)$.
By the chain rule, the probability density of $P(g(x))$ is $P'(g(x))g'(x)=p(g(x))\left|g'(x)\right|$.
If $g$ is monotonically decreasing, then $P(g(x))$ is the probability that the variable will be greater than $g(x)$.
By the chain rule, the probability density of $P(g(x))$ is $-P'(g(x))g'(x)=p(g(x))\left|g'(x)\right|$.
The situation is more complicated if $g$ is not monotonic; we need to sum the expressions above for all the points where $g(x)$ is a certain value.