Compute $\mathbb{E}_X \log(1+\exp(X))$ where $X \sim \mathcal{N}(\mu,\sigma^2)$

expected valuenormal distributionprobabilityprobability distributions

Problem

Obtain: $\mathbb{E}_X \log(1+\exp(X))$ for $X \sim \mathcal{N}(\mu,\sigma^2)$.


Definitions

  • Logistic function: $\displaystyle \mathcal{L}(x) := \frac{1}{1+\exp(-x)}$
  • Logit function: $\displaystyle \mathcal{L}^{-1}(y) := \log \left( \frac{y}{1-y} \right)$

Try

First let $\displaystyle Y\equiv -\log \left( \frac{1}{1+\exp(X)} \right) = -\log\left(\mathcal{L} (-X)\right)$. By change-of-variable,

$$
\begin{aligned}
f_Y(y) &= f_X(x) \left\vert \frac{\partial x}{\partial y} \right\vert\\[8pt]
&= \frac{1}{\sigma\sqrt{2\pi}} \exp\left( – \frac{(x-\mu)^2}{2\sigma^2} \right) \vert 1 + \exp(-x) \vert \\[8pt]
&= \frac{1}{\sigma\sqrt{2\pi}} \exp\left( – \frac{(\mathcal{L}^{-1}\left[ \exp(-y) \right]+\mu)^2}{2\sigma^2} \right) \frac{I(y>0)}{1-\exp(-y)} \ \ \because x = -\mathcal{L}^{-1}\left[ \exp(-y) \right] \\[8pt]
\end{aligned}
$$

Next, let us take the expectation

$$
\begin{aligned}
\mathbf{E}_Y[Y] &= \int_0^\infty y f_Y(y) dy \\[8pt]
&= \frac{1}{\sigma\sqrt{2\pi}} \int_0^\infty \frac{y}{1-\exp(-y)} \exp\left( – \frac{(\mathcal{L}^{-1}\left[ \exp(-y) \right]+\mu)^2}{2\sigma^2} \right) dy
\end{aligned}
$$

Let $w:=-\mathcal{L}^{-1}\left[ \exp(-y) \right]$, then

$$
dw = \exp(-y) \left( \frac{1}{\exp(-y)} + \frac{1}{1-\exp(-y)} \right) dy = \frac{dy}{1-\exp(-y)}
$$

which circularly begs the original question as

$$
\begin{aligned}
\mathbf{E}_Y[Y] &= \frac{1}{\sigma\sqrt{2\pi}} \int_{-\infty}^\infty \log(1+\exp(w)) \exp \left( -\frac{(w-\mu)^2}{2\sigma^2} \right) dw \\[8pt]
&= \mathbb{E}_X \log(1+\exp(X))
\end{aligned}
$$

Best Answer

This question: Expected Value of the Logarithm might answer your problem.

Here it states that

$$\mathbb{E}[\log(1+Y)]\approx \log(1+\mathbb{E}[Y])-\frac{\operatorname{Var}(Y)}{2(1+\mathbb{E}[Y])^2}$$ In our case, since $Y = e^X \sim \operatorname{LogNormal(}\mu,\sigma^2)$, we get that $$\mathbb{E}[\log(1+e^X)] \approx \log(1+e^{\mu+\frac12 \sigma^2})-\frac{(e^{\sigma^2}-1)e^{2 \mu + \sigma^2}}{2(1+e^{\mu+\frac12 \sigma^2})^2}$$

Related Question