First, I have read this post, this post and this post. All have very useful information. I have three other more specific questions.
I have estimated a negative binomial model using the glm.nb function of MASS and discovered the following parameters
Theta: 9.0487, S.E: 0.444
- Is it correct to assume that dispersion parameter has a standard deviation of 20.38?
- Does this value correspond to the Poisson overdispersion that is corrected by the negative binomial model or is my model still overdispersed?
- Joseph Hilbe states in his book that R's glm.nb function employs an inverted relationship of the dispersion parameter, theta. Thus a Poisson model results when theta approaches infinity. Suppose now that my second glm.nb model had estimates of Theta: 19.0487, S.E: 0.444. Would this model be less overdispersed than the first model?
Best Answer
Standard error is the standard deviation of an estimate of a parameter (see eg Wikipedia). So the standard deviation of the estimate $\hat\theta$ of the dispersion parameter $\theta$ is $0.444$. The ratio of an estimate to its standard error $\hat\theta/SE(\hat\theta)=20.38$ is often used as a test statistic for the null-hypothesis $\theta=0$. A large value here suggests that this null-hypothesis would probably be rejected. However,
as you noted, $\theta\rightarrow\infty$ corresponds to no overdispersion, so testing $\theta=0$ is not very meaningful
your third link discusses that testing for overdispersion has to be done carefully, especially in the context of the negative binomial distribution as the null-hypothesis is at the edge of the parameter space (whatever way it is parameterized).
The function fits a negative binomial distribution, and $\theta$ is one of its parameters. The negative binomial distribution is always overdispersed compared to the Poisson distribution (unless $\theta=\infty$). Since by modifying $\theta$ for a fixed $\mu$ the negative binomial distribution can achieve any variance ($Var(NB) = \mu +\frac{\mu^2}{\theta}$), there is no such thing as "overdispersion relative to the negative binomial distribution". Of course, it is possible that the negative binomial does not provide a good fit to the data, but the concept of overdispersion does not apply.
If the means are the same, then yes, if you have a larger $\theta$, then that model is less overdispersed compared to the Poisson distribution. If you change $\mu$ as well, then you have to think about quantifying overdispersion. For the Poisson distribution $E(X)=Var(X)$, but do you want to quantify deviations as $Var(X)/E(X)$? or $Var(X)-E(X)$? or some other way? Your question will have different answers for different ways to quantify overdispersion.