Solved – How to compute confidence intervals for maximum likelihood from the Fisher information matrix

confidence intervalfisher informationgeneralized linear modelmaximum likelihood

Looking at the Fisher information matrix for a simple linear model, such as here, I do not understand how to use the matrix to compute confidence intervals. There are multiple examples on the internet showing how to obtain the matrix, but I suppose that since it is thought to be trivial, it is nowhere shown how to use it.

My understanding after some reading, is that confidence intervals for a parameter under a normal distribution should be computed from $CI_{\theta} = \theta \pm 1.96 * \sqrt{\frac{1}{I(\theta)}}$.

For example, if I wanted to compute the 95% confidence interval for $\beta_1$ in the model $yi=\beta_0+\beta_1x_i+\epsilon_i$, how should I proceed given the matrix $\left[
\begin{matrix}
\tfrac{n}{\sigma^2} & \tfrac{\sum_i^n x_i}{\sigma^2} & 0\\
\tfrac{\sum_i^n x_i}{\sigma^2} & \tfrac{\sum_i^n x_i^2}{\sigma^2} & 0\\
0 & 0 & \tfrac{n}{2\sigma^4}
\end{matrix}
\right]$?

Best Answer

You have three parameters in your problem, so $\theta = (\beta_0, \beta_1, \sigma^2)$. $I(\theta)$ is a matrix and you cannot "divide by" $I(\theta)$, as in the formula in your second paragraph.

What you need instead is to take the inverse of $I(\theta)$. If you are looking for the confidence interval e.g. of $\beta_1$, you would take the element $[2,2]$ of $I(\theta)^{-1}$. and plug it in place of $1/I(\theta)$ in your formula. You would have then

$$CI_{\beta_1} = \hat\beta_1 \pm 1.96\sqrt{{I(\theta)^{22}}}$$

where $I(\theta)^{22}$ denotes the element $[2,2]$ of $I(\theta)^{-1}$.