[Math] Maximum likelihood estimate for 1/p in Binomial distribution

maximum likelihoodprobabilityprobability theorystatistical-inferencestatistics

Consider a discrete random variable $X$ with the binomial distribution $b(n,p)$ where $n$ is the number of Bernoulli trials and $p\in(0,1)$ is the success probability. The probability mass function for $X$, i.e., the number of successes in $n$ trials, is given by
$$ P(X=x)={n \choose x} p^x(1-p)^{n-x},\quad x=0,1,\ldots,n$$

The maximum likelihood estimate (MLE) for $p$ is given by
$$\widehat{p} = \frac{x}{n}$$
if one observes the event $X=x$.

My questions are the following:

  1. Can we compute the MLE for $1/p$ as follows:
    $$\widehat{\frac{1}{p}} = \frac{n}{x}$$
    using the invariance property of the MLE?

  2. It seems that the above estimator has infinite mean and variance for any finite $n$ since we have $X=0$ with probability $(1-p)^n$. Does this disturb asymptotic consistency, unbiasedness, and efficiency properties of the MLE?

Best Answer

So, suppose that we are Martians and know nothing about the binomial distribution; we know only that we have a parameter $q\geq 1$ and a formula describing the following probabilities

$$P(X=i)=\binom niq^{-i}\left(1-\frac1q\right)^{n-i}.\tag 1$$ ($i=0,1,\cdots, n.$) Now, assume that the outcome of our experiment is $X=0$.

Surprisingly, we are familiar with the maximum likelihood method. So, we apply it. We have to find the $q$ that maximizes

$$\left(1-\frac1q\right)^n.$$

Apparently, for any finite $q$ there is a better one. That is $q=\infty$ seems to be the maximum likelihood estimate.

Now, we suddenly learn what the binomial distribution is. We immediately conclude that $p=0$ is the solution for the "true earthly parameter." Away we sail then immediately.


EDIT

Let's try to find the maximum likelihood parameter $q\geq1$ in the case of $n$ experiments and $i$ successful outcomes assuming that the distribution is given by $(1)$. We can forget about the multiplier $\binom ni$. So, after dividing $(1)$ by $\binom ni$ take the derivative of $(1)$ with respect to $q$. And set the derivative equal to zero then solve the equation for $q$.

Here is the equation

$$(n-i)q^{-i-2}\left(1-\frac1q\right)^{n-i-1}=iq^{-i-1}\left(1-\frac1q\right)^{n-i}.$$

We will have to exclude $q=1$ from now on. However $q=1$ is certainly the solution for $n=i$. Divide both sides by $q^{-i-1}\left(1-\frac1q\right)^{n-i}$. The resulting equation is

$$(n-i)q^{-1}\left(1-\frac1q\right)^{-1}=i.$$

from here we get the expected result:

$$\hat q=\frac ni.$$

NOTE

You can see here that the MLE does have the invariance property. So it is true that if $\frac in$ is the MLE for $p$ then for $q=\frac1p$ the MLE is $\frac ni$. I did the proof above for you and I because I don't believe if theorems (invariance property this time) whose proof I've never digested.

Related Question