bernoulli-distributionestimationestimatorsunbiased-estimator
Let $X_1, X_2, …, X_n \sim Bernoulli(p)$, where $n > 5$. Let $\hat p = \frac{1}{5} \sum_{i=1}^{5} X_i$ be the estimator of $p$. Is it biased?
My guess is no, since $E[\hat p] = \frac{1}{5} \sum_{i=1}^{5} E[X_i] = \frac{1}{5} \sum_{i=1}^{5} p = \frac{5p}{5} = p$.
However, I find this counterintuitive. If I flip a fair coin 5 times to compute the estimator and happen to get heads ($X_i = 0$) every time, then $\hat p = 0$, but the real $p$ is $0.5$, so I am underestimating the parameter $p$.
$\hat{p}$ converges almost surely to $p$ by SLLN. Since the product of two almost-sure convergent random variable converges almost-surely to the product of their limits, it follows that $s^2$ converges almost surely to $p(1-p)$, which also implies it converges in probability, hence consistency.
I hAVE tried to find the conditional distribution for two cases only...you should try to find the conditional distribution for other two cases.It will clear your doubt.
Best Answer
Unbiased does not mean that you estimate the parameter correctly in each and every sample, it means that the errors you make in any particular samples tend to average eachother away across many independent samples.
In your case, every sample in which you flip no heads is matched by an equally likely sample where you flip no tails. In the long run you would get about as many no heads samples as no tails, so the number of heads across all such samples averages to 2.5, and the proportion of heads in such samples averages out to $\frac{2.5}{5} = 0.5$
It's consistent. Your estimator $\hat p$ is a sample mean, and you can calculate the variance of a sample mean quite easily (as long as the flips are independent)
$$ var(\hat p) = \frac{5p(1 - p)}{n} $$
As your sample get's larger, $n \rightarrow \infty$, and so $var(\hat p) \rightarrow 0$. Using Chebyshev's inequality immediately implies that the estimator is consistent.