[Math] Find the MLE and asymptotic variance

maximum likelihoodvariance

Let $X_1,\dots X_n$ be an i.i.d sample from the distribution with frequency function $$P(X=-1) = {\theta}/2 = P[X=1] \qquad \text{and}\qquad P[X=0] = 1-\theta$$
Note that the frequency function can be expressed as $$P(X=x) = \left(\frac{\theta}{2}\right)^{|x|}(1-\theta)^{1-|x|}$$ for $x = -1,0,1$.

  1. Find the MLE of $\theta$.
  2. What is the exact variance of the MLE.
  3. Find the asymptotic variance of the MLE.

I don't even know how to begin doing question 1. As for 2 and 3, what is the difference between exact variance and asymptotic variance?

Best Answer

$1.$ The likelihood $f(x_1,\dots,x_n;θ)$ of the sample $X_1,\dots,X_n$ is equal to $$f_{X_1,\dots,X_n}(x_1,\dots,x_n;θ)=f_{X_1}(x_1;θ)\dots f_{X_n}(x_n;θ)=\left(\frac{\theta}{2}\right)^{\sum_{i=1}^n|X_i|}(1-\theta)^{n-\sum_{i=1}^n|X_i|}$$ Take the $\ln$ to simplify $$\ln f(x_1,\dots,x_n;θ)=\sum_{i=1}^n|X_i|\cdot\ln\left(\frac{\theta}{2}\right)+\left(n-\sum_{i=1}^n|X_i|\right)\cdot\ln (1-\theta)$$ Differentiate with respect to $\theta$ $$\frac{d}{d\theta}\ln f(x_1,\dots,x_n;θ)=\sum_{i=1}^n|X_i|\cdot \frac{1}{θ}+\left(n-\sum_{i=1}^n|X_i|\right)\cdot\frac{1}{\theta-1}$$ and set the derivative equal to $0$ to find the $\hat \theta$ that maximizes the likelihood \begin{align}\sum_{i=1}^n|X_i|\cdot \frac{1}{\hat θ}+\left(n-\sum_{i=1}^n|X_i|\right)\cdot\frac{1}{\hat\theta-1}&=0\iff\\(\hat\theta-1)\sum_{i=1}^n|X_i|+\left(n-\sum_{i=1}^n|X_i|\right)\cdot\hat\theta&=0\iff \\[0.2cm]\hat \theta&=\frac1n\sum_{i=1}^n|X_i|\end{align}

$2.$ Now you can calculate the variance of $\hat\theta$ since $\hat\theta$ is a function of the random variables $X_i, i=1,\dots,n$ and as such a random variable itself

\begin{align}Var(\hat\theta)&=Var\left(\frac1n\sum_{i=1}^n|X_i|\right)\\[0.2cm]&=\frac1{n^2}\sum_{i=1}^nVar\left(|X_i|\right)=\frac1{n^2}\sum_{i=1}^nE|X_i|^2-\left(E|X_i|\right)^2\\[0.2cm]&=\frac{1}{n^2}\sum_{i=1}^n\left(1^2\cdot\theta+0^2\cdot(1-\theta)\right)-\left(1\cdot\theta+0\cdot(1-\theta)\right)^2\\[0.2cm]&=\frac{1}{n^2}\sum_{i=1}^n\theta-\theta^2=\frac{1}{n^2}n\theta(1-\theta)=\frac{1}{n}\theta(1-\theta)\end{align}

$3.$ Now, let $n\to+\infty$ to find the asymptotic variance of the MLE, i.e. the variance when you take a sample of size $n$ with $n\to+\infty$ (a huge sample) which is $$\lim_{n\to+\infty}Var(\hat\theta)=\lim_{n\to+\infty}\frac{1}{n}\theta(1-\theta)=0$$

Related Question