Let $X \sim \mathrm{Gamma}(\alpha, \theta),$ where $$f(x) = \frac {x^{\alpha – 1} e^{-\frac x \theta}} {\theta^{\alpha}\Gamma(\alpha)}.$$ The log-likelihood function can be shown to be $$l(\alpha, \theta) = -n\alpha\ln\theta – n\ln[\Gamma(\alpha)] + (\alpha – 1)\sum^n_{i = 1} \ln x_i – \theta^{-1}\sum^n_{i = 1} x_i.$$
Now, suppose we reparameterise $X$ by introducing $$\mu = \alpha\theta$$ and the log-likelihood function can be shown to be $$l(\alpha, \mu) = n\alpha\ln\alpha -n\alpha\ln\mu – n\ln[\Gamma(\alpha)] + (\alpha – 1)\sum^n_{i = 1} \ln x_i – \alpha\mu^{-1}\sum^n_{i = 1} x_i.$$
I am interested in deriving the Fisher information matrix for the reparameterised Gamma distribution which was given to me as
$$\begin{aligned}
I(\alpha, \mu) & = –
\begin{bmatrix}
\mathbb{E}_{\alpha, \mu}\left(\frac {\partial^2l(\alpha, \mu)} {\partial\alpha^2} \right) & \mathbb{E}_{\alpha, \mu}\left(\frac {\partial^2l(\alpha, \mu)} {\partial\alpha\partial\mu} \right) \\
\mathbb{E}_{\alpha, \mu}\left(\frac {\partial^2l(\alpha, \mu)} {\partial\alpha\partial\mu} \right) & \mathbb{E}_{\alpha, \mu}\left(\frac {\partial^2l(\alpha, \mu)} {\partial\mu^2} \right)
\end{bmatrix} \\
& =
\begin{bmatrix}
n\alpha^{-1}\left(\alpha\frac {d\psi(\alpha)} {d\alpha} – 1\right) & 0 \\
0 & n\alpha\mu^{-2}
\end{bmatrix},
\end{aligned}$$
where $\psi(\alpha)$ is the digamma function.
In particular, I am having issues deriving $$I_{(2, 2)}(\alpha, \mu) = n\alpha\mu^{-2}.$$ It is trivial to obtain the relevant first and second order derivatives of the reparameterised log-likelihood function which are $$\frac {\partial l(\alpha, \mu)} {\partial\mu} = -n\alpha\mu^{-1} + \alpha\mu^{-2}\sum^n_{i = 1} x_i$$ and $$\frac {\partial^2l(\alpha, \mu)} {\partial\mu^2} = n\alpha\mu^{-2} – 2\alpha\mu^{-3}\sum^n_{i = 1} x_i.$$
However, I am unsure how taking the expectation of my last expression above will give $-n\alpha\mu^{-2}$ such that it reconciles with the result I need.
Any intuitive explanations will be greatly appreciated!
Best Answer
Let $P_{\theta}(dw)=e^{\ell_w(\theta)}\nu(dw)$ be a Fisher model with $\theta\in \Theta\subset R^d$. Let $$I(\theta)=\int_{\Omega}\ell'_{w}\theta)\otimes \ell'_{w}(\theta)P_{\theta}(dw)=-\int_{\Omega} \ell''_{w}(\theta)P_{\theta}(dw)$$ be the $d\times d $ Fisher information matrix. Let $t \mapsto \theta=f(t)$ from $\Theta_1\subset R^d$ to $\Theta$ be a reparameterization of the previous Fisher model. Then the Fisher information of the new model is $$J(t) =f'(t)^TI(f(t))f'(t),$$ where $f'(t)$ is the Jacobian matrix of $t\mapsto f(t).$ Proof: just compute.