Wikipedia states that the expected value of a normally distributed variable $x>a$ can be written as
\begin{equation}
E(x|x>a) = \mu + \sigma \frac{\phi(\alpha)}{1 -\Phi(\alpha)},
\end{equation}
where $\alpha = (a – \mu)/\sigma$ and $\phi(\alpha),\Phi(\alpha)$ are the pdf and cdf of the standard normal distribution.
The article cites a book when giving these equations and I looked it up, but the book also does not give a derivation for the expression.
I have been wondering how general this expression is and if we can draw some conclusions for different distribution functions, e.g. that $E(x|x>a)$ is an increasing function of $\mu$. I would therefore like to see a derivation to see which of the steps are due to $x$ being normally distributed and which insights generalize. Can you provide me with a derivation?
Best Answer
Let $Y=X\mid X>a$. Then the PDF of $Y$ is $$f_Y(x)=[x>a]\cdot\frac{\phi\left(\frac{x-\mu}\sigma\right)}{\sigma \cdot (1-\Phi(\alpha))}.$$
Hence, \begin{equation}\label 1\tag 1\mathbb E Y=\int_{-\infty}^\infty x\cdot f_Y(x)\,\mathrm dx=\frac1{\sigma \cdot (1-\Phi(\alpha))}\int_a^\infty x\cdot\phi\left(\frac{x-\mu}\sigma\right)\,\mathrm dx.\end{equation}
We will use the (obvious) substitution $y=\frac{x-\mu}\sigma$. Then $x=\sigma y+\mu$ and thus $\mathrm dx\sim\sigma \mathrm dy$. Hence, we get (note that the lower integration bound changes from $x=a$ to $y=\alpha$) \begin{align}\label 2\tag 2 \int_a^\infty x\cdot\phi\left(\frac{x-\mu}\sigma\right)\,\mathrm dx&=\int_\alpha^\infty (\sigma y+\mu)\cdot\phi(y)\cdot\sigma\,\mathrm dy\\& =\sigma^2\int_\alpha^\infty y\cdot\phi(y)\,\mathrm dy+\color{blue}{\sigma\mu\int_\alpha^\infty\phi}\\ &=\sigma^2\int_\alpha^\infty y\cdot\phi(y)\,\mathrm dy+\color{blue}{\sigma\mu\cdot(1-\Phi(\alpha))}. \end{align}
Now, about the first integral (I will use the substitution $u=\frac12y^2$, $\mathrm du\sim y\,\mathrm dy$): \begin{align}\tag 3\label 3 \int_\alpha^\infty y\cdot\phi(y)\,\mathrm dy&=\int_\alpha^\infty y\cdot\exp\left(-\frac12y^2\right)\,\mathrm dy\\ &= \int_{\frac{\alpha^2}2}^\infty \exp(-u)\,\mathrm du \\ &=[-\exp(-u)]^\infty_{\frac{\alpha^2}2} \\&= 0+\exp\left(-\frac{\alpha^2}2\right)=\phi(\alpha). \end{align}
Hence, by \eqref{2}, $$\int_a^\infty x\cdot\phi\left(\frac{x-\mu}\sigma\right)\,\mathrm dx=\sigma^2\phi(\alpha)+\sigma\mu\cdot(1-\Phi(\alpha)).$$
Using this in \eqref{1} gives the desired result.