Let $ X \sim \mathcal{N} ( \mu, \sigma^2 ) $, $ – \infty \leqslant a < b \leqslant +\infty $ ($ a, b \ne \infty $ simultaneously) and $ Y $ has a truncated normal distribution on $ (a, b )$, i.e. $ Y $ has pdf
$ \frac{1}{\Phi(\frac{b – \mu}{\sigma}) – \Phi(\frac{a – \mu}{\sigma}) } \frac{1}{\sigma}\phi(\frac{x – \mu}{\sigma}) \mathbf{1} ( a < x < b ) ,$
where $ \phi(x)$ and $ \Phi(x) $ are pdf and cdf of standard normal distribution.
How to prove that the variance of truncated normal distribution is always less than the variance of untruncated distribution, i.e.
$ \mathrm{Var} Y < \mathrm{Var} X $?
Here is a neat proof that $ \mathrm{Var} Y \leqslant \mathrm{Var} X $, but I can't extend it to cover the equality case.
Also there is a proof in this article, but it seems wrong, because they only assumed $ \mu \in (a, b) $.
Update:
Now I think it can be proved in two steps.
1) For one-sided truncations the strick formula $ \mathrm{Var} Y' < \mathrm{Var} X $ follows from estimations of inverse Mills ratio.
2) For two-sided truncations $ \mathrm{Var} Y \leqslant \mathrm{Var} Y' < \mathrm{Var} X $, because one-sided truncated distributions is log-concave.
Is it correct proof or am I missing something?
Best Answer
Since the OP was kind enough to reference an older answer of mine, and also to alert me to the fact, I will provide some input here, of naive flavor, I guess, since mathoverflow is definitely over and out of my league. I will use $\Phi()$ for the standard normal CDF and $\phi()$ for the standard normal PDF. In my answer in math.se I had proved that
$${\rm Var}(Y) = {\rm Var}(X)\cdot \left[1+\sigma^2\frac{\partial^2 \ln H(\mu)}{\partial \mu^2}\right],\;\; -1 <\sigma^2\frac{\partial^2 \ln H(\mu)}{\partial \mu^2} \leq 0 $$
where:
$$\ln H(\mu)=\ln \big(\Phi(\beta(\mu))-\Phi(\alpha(\mu))\big)$$
$$\alpha=(a-\mu)/\sigma, \;\beta=(b-\mu)/\sigma$$.
The weak inequality comes from the fact that $H$ is log-concave (see the original post).
Calculating the second derivative, in order to prove strict inequality, we have to show that
$$\frac{\partial^2 \ln H(\mu)}{\partial \mu^2} < 0 \implies D \equiv [-\phi'(\alpha)+\phi'(\beta)][\Phi(\beta)-\Phi(\alpha)]-[\phi(\alpha)-\phi(\beta)]^2 < 0 $$
We note that $\Phi(\beta)-\Phi(\alpha) >0 $ always.
CASE A : $a \leq \mu \leq b$ (including one-sided trunctaions).
Here, by the unimodality of $\phi()$, with $\mu$ being the mode, we have that
$$\phi'(\alpha) \geq 0, \phi'(\beta) \leq 0 \implies -\phi'(\alpha)+\phi'(\beta) < 0$$
since the two derivatives cannot be both equal to zero.
Then $D < 0$ always (including truncation symmetric around the mean, where, due to the fact that $\phi()$ is an even function, we have that $\phi(\alpha)=\phi(\beta)$, and the second element of $D$ will be zero. But the first element is always strictly negative).
So for this case we have proved that $\partial^2 \ln H(\mu)/\partial \mu^2 < 0$ as we wanted.
The cases left out are all the cases where $\mu$ does not belong to the (two-sided or one-sided) truncated support.
CASE B : Truncated support $S_B\equiv (-\infty , b], \mu \notin S_B$
Here the inequality to prove is
$$D_B \equiv \phi'(\beta)\Phi(\beta)-\phi(\beta)^2 < 0$$
Since $\phi'(\beta) = -\beta \phi(\beta)$ we wan to show
$$-\beta \phi(\beta)\Phi(\beta)-\phi(\beta)^2 < 0 \implies -\beta \Phi(\beta)-\phi(\beta) < 0 \implies \beta \Phi(\beta)+\phi(\beta) > 0$$
But this holds because
$$\beta \Phi(\beta)+\phi(\beta) = \int_{-\infty}^{\beta}\Phi(t){\rm d}t$$
and the integral is necessarily strictly greater than zero since $\Phi()$ is non-negative and non-constantly zero. So we have proved here too what we needed to prove.
CASE C : Truncated support $S_C\equiv [a, \infty ), \mu \notin S_C$
Here the inequality to prove is
$$-\phi'(\alpha)[1-\Phi(\alpha)]-\phi(\alpha)^2 < 0 \implies \alpha \phi(\alpha)[1-\Phi(\alpha)]-\phi(\alpha)^2 < 0 $$
and simplifying and using the symmetry properties of the two functions we have
$$ \implies (-\alpha)\Phi(-\alpha) + \phi(-\alpha) >0$$ and we are at the same situation as in Case B. So QED here too.
I am not treating the two-sided truncation cases $a < b < \mu$ and $\mu < a < b$. As with the cases I treated, I am certain there exists some more advanced and elegant way to prove what we want.