In Probability and Random Processes (Grimmett and Stirzaker, 3rd ed.) the following proof is offered for LOTUS for the case that $g: \mathbb{R}\rightarrow\mathbb{R^+}$.
Lemma. If $X$ has density function $f$ with $f(x) = 0$ when $x < 0$,
$E[X] = \int_0^{\infty} P(X>x)\ dx$
Proof of LOTUS.
Step 1. Using the lemma,
$E[g(X)] = \int_0^{\infty} P(g(X)>x)\ dx = \int_0^{\infty} (\int_B f_X(y)\ dy)\ dx$
where $B = \{y:g(y) > x\}$.
Step 2. Change the order of integration.
$E[g(X)] = \int_0^{\infty}\int_0^{g(y)}\ dxf_X(y)\ dy = \int_0^{\infty}g(y)f_X(y)dy$
My questions:
1- Going from step 1 to step 2, how are the bounds of the integral determined? From definition of B, it seems to me that $g(y)$ can be above or below the $x=y$ line so not clear how the integration limits were cleanly determined.
2- How to prove this for the case where $g$ can be positive or negative? (i.e., $g: \mathbb{R}\rightarrow\mathbb{R}$)
Best Answer
For your question (1): From Step 1 to Step 2 there's an interchange of the order of integration. In step 1 the inner integral is over $y$ while in step 2 the inner integral over $x$. The interchange can be seen more clearly if we write $$\int_B f_X(y)\,dy = \int_{y=0}^\infty I[g(y)>x]f_X(y)\,dy$$ where $I(A)$ is the indicator of event $A$. With this formulation, we can interchange using Fubini's theorem (note the integrand is nonnegative): $$\int_{x=0}^\infty\left(\int_{y=0}^\infty I[g(y)>x]f_X(y)\,dy\right)\,dx =\int_{y=0}^\infty\left(\int_{x=0}^\infty I[g(y)>x]f_X(y)\,dx\right)\,dy $$ The inner integral on the RHS can now be written $$\int_{x=0}^{g(y)}f_X(y)\,dx=g(y)f_X(y).$$
For your question (2), you handle a $g$ that can take both positive and negative values by decomposing $g$ into its positive and negative parts: $$g(x) = g^+(x) - g^-(x)$$ where we define $g^+$ to be: $$g^+(x) := \begin{cases}g(x) & \text{if $g(x)>0$}\\0 &\text{if $g(x)\le0$}\end{cases}$$ and similarly $g^-$ is the negative part: $$g^-(x) := \begin{cases}0 & \text{if $g(x)>0$}\\-g(x) &\text{if $g(x)\le0$}\end{cases}$$ Then both $g^+$ and $g^-$ are functions from $\mathbb R$ into $\mathbb R^+$ and you can apply the previous result to each: $$ E(g(X))=E[g^+(X)-g^-(X)]=\int g^+(y)f_X(y)dy-\int g^-(y)f_X(y)dy=\int g(y)f_X(y)dy.$$