Gamma Distribution – Expected Value of the Logarithm of Gamma Distribution

expected valuegamma distribution

If the expected value of $\mathsf{Gamma}(\alpha, \beta)$ is $\frac{\alpha}{\beta}$, what is the expected value of $\log(\mathsf{Gamma}(\alpha, \beta))$? Can it be calculated analytically?

The parametrisation I am using is shape-rate.

Best Answer

This one (maybe surprisingly) can be done with easy elementary operations (employing Richard Feynman's favorite trick of differentiating under the integral sign with respect to a parameter).


We are supposing $X$ has a $\Gamma(\alpha,\beta)$ distribution and we wish to find the expectation of $Y=\log(X).$ First, because $\beta$ is a scale parameter, its effect will be to shift the logarithm by $\log\beta.$ (If you use $\beta$ as a rate parameter, as in the question, it will shift the logarithm by $-\log\beta.$) This permits us to work with the case $\beta=1.$

After this simplification, the probability element of $X$ is

$$f_X(x) = \frac{1}{\Gamma(\alpha)} x^\alpha e^{-x} \frac{\mathrm{d}x}{x}$$

where $\Gamma(\alpha)$ is the normalizing constant

$$\Gamma(\alpha) = \int_0^\infty x^\alpha e^{-x} \frac{\mathrm{d}x}{x}.$$

Substituting $x=e^y,$ which entails $\mathrm{d}x/x = \mathrm{d}y,$ gives the probability element of $Y$,

$$f_Y(y) = \frac{1}{\Gamma(\alpha)} e^{\alpha y - e^y} \mathrm{d}y.$$

The possible values of $Y$ now range over all the real numbers $\mathbb{R}.$

Because $f_Y$ must integrate to unity, we obtain (trivially)

$$\Gamma(\alpha) = \int_\mathbb{R} e^{\alpha y - e^y} \mathrm{d}y.\tag{1}$$

Notice $f_Y(y)$ is a differentiable function of $\alpha.$ An easy calculation gives

$$\frac{\mathrm{d}}{\mathrm{d}\alpha}e^{\alpha y - e^y} \mathrm{d}y = y\, e^{\alpha y - e^y} \mathrm{d}y = \Gamma(\alpha) y\,f_Y(y).$$

The next step exploits the relation obtained by dividing both sides of this identity by $\Gamma(\alpha),$ thereby exposing the very object we need to integrate to find the expectation; namely, $y f_Y(y):$

$$\eqalign{ \mathbb{E}(Y) &= \int_\mathbb{R} y\, f_Y(y) = \frac{1}{\Gamma(\alpha)} \int_\mathbb{R} \frac{\mathrm{d}}{\mathrm{d}\alpha}e^{\alpha y - e^y} \mathrm{d}y \\ &= \frac{1}{\Gamma(\alpha)} \frac{\mathrm{d}}{\mathrm{d}\alpha}\int_\mathbb{R} e^{\alpha y - e^y} \mathrm{d}y\\ &= \frac{1}{\Gamma(\alpha)} \frac{\mathrm{d}}{\mathrm{d}\alpha}\Gamma(\alpha)\\ &= \frac{\mathrm{d}}{\mathrm{d}\alpha}\log\Gamma(\alpha)\\ &=\psi(\alpha), }$$

the logarithmic derivative of the gamma function (aka "polygamma"). The integral was computed using identity $(1).$

Re-introducing the factor $\beta$ shows the general result is

$$\mathbb{E}(\log(X)) = \log\beta + \psi(\alpha)$$

for a scale parameterization (where the density function depends on $x/\beta$) or

$$\mathbb{E}(\log(X)) = -\log\beta + \psi(\alpha)$$

for a rate parameterization (where the density function depends on $x\beta$).

Related Question