This question is an extension to this previous question asked by myself:
When doing a maximum likelihood fit, we often take a ‘Gaussian
approximation’. This problem works through the case of a measurement from a Poisson distribution. The Poisson distribution results in a likelihood for the average number $\mu$, given that $n$ events were observed:$L(\mu;n)=\cfrac{\mu^{n}e^{-\mu}}{n!}$$\mu_0$ is defined to be the value of $\mu$ which gives the maximum of the likelihood, at which the likelihood has a value $L_0$.
Taking the natural logarithm of $L$ gives
$$\ln(L) = \ln(\mu^n)+\ln(e^{-\mu})-\ln(n!)=n\ln(\mu)-\mu – \ln(n!)\tag{1}$$
and taking the derivative gives
$$\frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}=\color{blue}{\frac{n}{\mu}-1}\tag{2}$$
The maximum is given when $(2)$ is equal to zero; so $\mu_0$ satisfies
$$\frac{n}{\mu_0}-1=0\implies \mu_0=n$$
Therefore $$\ln(L_0)=n\ln(n) – n -\ln(n!)\tag{3}$$
$\fbox{The objective in this question is to Taylor expand $\ln(L)$ to second order.}$
Performing a Taylor expansion up to the second order of $\ln(L)$ around the maximum, where $\mu_0=n$, is
$$\ln(L)\approx\ln(L_0) + \frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n(\mu-n) + \frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}\bigg|_n\frac{(\mu-n)^2}{2!}$$
Taking the second derivative gives
$$\frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}=\color{#180}{-\frac{n}{\mu^2}}\tag{4}$$
Hence, to second order
$$\ln(L)\approx \ln(L_0) – \frac{1}{n}\frac{(\mu-n)^2}{2!}$$
$$\implies\fbox{$\color{red}{\ln(L)\approx \ln(L_0) – \frac{(\mu-n)^2}{2n}}$}\tag{?}$$
I think the final answer marked $\color{red}{\mathrm{red}}$ is wrong.
So here is my attempt:
$$\ln(L)\approx\ln(L_0) + \underbrace{\frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n}_{\underbrace{=\color{blue}{\dfrac{n}{\mu}-\large{1}}}_{\Large\text{from (2)}}}\times(\mu-n) + \underbrace{\frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}\bigg|_n}_{\underbrace{=\color{#180}{-\dfrac{n}{\mu^2}\space}}_{\Large\text{from (4)}}}\times \frac{(\mu-n)^2}{2!}$$
$$\implies\ln(L)\approx\ln(L_0) + \left(\frac{n}{\mu}-1\right)\times (\mu-n) -\frac{n}{\mu^2}\times \frac{(\mu-n)^2}{2}$$
$$\implies\ln(L)\approx\ln(L_0) + \left(\frac{n-\mu}{\mu}\right)\times (\mu-n) -\frac{n(\mu-n)^2}{2\mu^2}$$
$$\implies\ln(L)\approx\ln(L_0)-\frac{(\mu-n)^2}{\mu}-\frac{n(\mu-n)^2}{2\mu^2}$$
$$\implies\color{#F80}{\ln(L)\approx\ln(L_0)-\frac{(\mu-n)^2}{\mu}\left(\frac{n}{2\mu}+1\right)}$$
So my question is why the expression marked $\color{#F80}{\mathrm{orange}}$ is not the equal to the expression marked $\color{red}{\mathrm{red}}$?
Could someone please explain what I am doing wrong?
Best Answer
The red answer is correct. The mistake in the orange answer is simply forgetting to evaluate the derivatives at the point $\mu = \mu_0 = n$ and instead using the general variable $\mu$ in these expressions.
The second order Taylor polynomial of $\ln(L)$ about $\mu=n$ is given by
$$\ln(L_0) + \color{red}{\left.\frac{d\ln L}{d\mu}\right|_{\mu=n}}(\mu-n) + \color{blue}{\left.\frac{d^2\ln L}{d\mu^2}\right|_{\mu=n}}\frac{(\mu-n)^2}{2}\\= \ln(L_0) + \color{red}{0}\cdot (\mu-n) + \left(\color{blue}{-\frac{1}{n}}\right)\frac{(\mu-n)^2}{2} = \ln(L_0) - \frac{(\mu-n)^2}{2n}$$
since
$$\left.\frac{d\ln L}{d\mu}\right|_{\mu=n} = \left(\frac{n}{\mu}-1\right)_{\mu=n} = 0,~~~~~~~~~\left.\frac{d^2\ln L}{d\mu^2}\right|_{\mu=n} = \left(-\frac{n}{\mu^2}\right)_{\mu=n} = -\frac{1}{n}$$