Calculating Parameter Confidence Intervals Using Hessian Matrix in R

maximum likelihoodr

Given an output from optim with a hessian matrix, how to calculate parameter confidence intervals using the hessian matrix?

fit<-optim(..., hessian=T)

hessian<-fit$hessian

I am mostly interested in the context of maximum likelihood analysis, but am curious to know if the method can be expanded beyond.

Best Answer

If you are maximising a likelihood then the covariance matrix of the estimates is (asymptotically) the inverse of the negative of the Hessian. The standard errors are the square roots of the diagonal elements of the covariance (from elsewhere on the web!, from Prof. Thomas Lumley and Spencer Graves, Eng.).

For a 95% confidence interval

fit<-optim(pars,li_func,control=list("fnscale"=-1),hessian=TRUE,...)
fisher_info<-solve(-fit$hessian)
prop_sigma<-sqrt(diag(fisher_info))
prop_sigma<-diag(prop_sigma)
upper<-fit$par+1.96*prop_sigma
lower<-fit$par-1.96*prop_sigma
interval<-data.frame(value=fit$par, upper=upper, lower=lower)

Note that:

  • If you are maximizing the log(likelihood), then the NEGATIVE of the hessian is the "observed information" (such as here).
  • If you MINIMIZE a "deviance" = (-2)*log(likelihood), then the HALF of the hessian is the observed information.
  • In the unlikely event that you are maximizing the likelihood itself, you need to divide the negative of the hessian by the likelihood to get the observed information.

See this for further limitations due to optimization routine used.

Related Question