Solved – Bernoulli random variable parameter estimation

bernoulli-distributionestimationmaximum likelihood

Suppose $\theta$ is the probability that a Bernoulli random variable is one (therefore $1-\theta$ is the probability that it's zero). I have a sequence of $n$ of these i.i.d. Bernoulli random variables, $m$ out of which are ones. It is a well-known fact that $\hat{\theta}=m/n$ is a maximum likelihood estimator (MLE) for $\theta$ (and it is also minimum-variance unbiased estimator).

Now, in my case, I know that $\theta=\frac{1}{2}+\frac{1}{2}e^{-x}$ for some unknown $x$, where $x$ satisfies $0<x<\infty$. This implies that $\theta$ satisfies $\frac{1}{2}<\theta<1$. I am trying to find an estimator for $x$ given $m$ and $n$ (and, perhaps the full vector of observed random variables). So, I try to find an MLE by first writing down the log-likelihood function:

$$\log\mathcal{L}(x;n,m)=-n\log 2+n\log(1-e^{-x})+m(\log(1+e^{-x})-\log(1-e^{-x}))$$

Then I take the derivative with respect to $x$ and set it to zero:

$$\frac{\partial\log\mathcal{L}(x;n,m)}{\partial x}=\frac{n}{e^x-1}-\frac{2m}{e^{-x}-e^{x}}=0$$

Solving for $x$ yields the estimator $\hat{x}=-\log(2m/n -1)$. However, if less than half the random variables are one (i.e. $m/n<1/2$ — and this certainly can happen in the scenario that I described), then the estimator yields an imaginary number (due to a negative in the logarithm).

What did I do wrong? How does one estimate $x$ in this case?

EDIT

John A. Ramey pointed out that how for certain values of $\theta$, MLE leads to a degenerate answer. Perhaps there are methods other than MLE?

Best Answer

Instead of tedious derivations, simply invoke the invariance property of MLEs. Then solve for $x$ using basic algebra. However, note that this approach will lead to the same estimator you derived, i.e., $\hat{x}=-\log(2m/n -1)$.

So what to do? First, ignore estimation for the moment. Look at what the true value of $x$ would be if you knew the true value of $\theta$.

Suppose $\theta = 0$. Then, $e^{-x} = -1$, which implies that $x = - \log (-1)$. As you noted, this is not a real value.

So what about $\theta = 1/2$? We have that $e^{-x} = 0$, which is a problem for you yet again.

The point here is that with a couple of special cases based on known values, we can see that $x$ is undefined in a large number of cases. Your estimation is not wrong per se. The value of $x$ is simply undefined (not real) for certain values of $\theta$.

Related Question