So I would like to derive the score function for my GLM, which in this case happens to be logistic regression. My approach is to write the pdf as an exponential family, I get it from these slides.
I can write the pdf as, where $\theta=\log{\frac{p}{1-p}}$, $b(\theta)=-n\log{(1-p)}$, $a(\phi)=1$ and $c(y,\phi)=\log{\binom{n}{p}}$.
$$\frac{y\theta-b(\theta)}{a(\theta)}+c(y,\theta)$$
As far as I can see I should get, when I differentiate it with regards to $\theta$:
$$\frac{y-\theta}{1}$$
But I cannot really get that result, I instead get $b'(\theta)=\frac{n}{1-p}$, so is my expectations wrong or am I differentiating something wrong? And what would the score function of a logistic regression look like?
EDIT
So from these notes. I get that the score function of a GLM with the following link function $g(\mu)=\log{\frac{\mu}{1-\mu}}$, for the $i$th observation will be:
$$ \frac{y_i-\mu_i}{a(\phi_i)}\frac{1}{b^{\prime\prime}(\theta_i)}\frac{x_i}{g'(\mu_i)} $$
And since $$g'(\mu)=\frac{\mu-1}{\mu(1-\mu)}$$
And $$b^{\prime\prime}(\theta)=\frac{ne^{\eta}}{(e^{\eta}+1)^2}$$
Then I would get that the the score function for a logistic regregssion GLM is:
$$\frac{y_i-\mu_i}{\frac{ne^{\eta_i}}{(e^{\eta_i}+1)^2}}\frac{x_i}{\frac{\mu_i-1}{\mu_i(1-\mu_i)}}$$
So are my inferences correct? And can this statement be made prettier?
As $g'(\mu)=1$ and $b^{\prime\prime}(\theta)=1$ (since $b'(\theta)=\frac{mu^2}{2}$ and $\theta=\mu$) for a GLM where the link function is just $g(\mu)=\mu$, so this formula also holds for getting the score function of a LM right?
Best Answer
Ok so the main thing is $\theta=\ln\frac{p}{1-p}$, but $b(\theta)$ isn't actually a function of $\theta$ yet! so you need to rearrange for $\theta$ in terms of $p$. First take natural logs: $$e^\theta=\frac{p}{1-p}$$ Therefore, $$p=\frac{e^\theta}{1+e^\theta}$$ Now sub that into $b(\theta)$, so $$b(\theta)=-n(ln(1-\frac{e^\theta}{1+e^\theta}))$$ So now $b'(\theta)$ is $$\frac{ne^\theta}{e^\theta+1}$$ Now sub the $p=\frac{e^\theta}{e^\theta+1}$ back in $$np$$ which is the mean of the binomial distribution.