I am struggeling to understand the how an estimator is arrived and whether it can be determined it is biased or not.
I have this example
Let $X_1 , X_2 ,\ldots, X_7$ denote a random sample from a population having mean $\mu$ and variance $\sigma^2$. Consider the following estimator of $\mu$:
$$\hat{\Theta}_1 = \frac{X_1 + X_2 +\cdots+ X_7}{7}$$
$$\hat{\Theta}_2 = \frac{2X_1 − X_6 + X_4}{2}$$
a) Is either Estimator unbiased?
So since these are trying to estimate the $\mu$, $E[\hat{\Theta}] = \mu$.
So i've done it like this
$E[\hat{\Theta}_1] = \frac{E[X_1] + E[X_2]+\cdots+E[X_7]}{7}$.
And since the expected value has been taken from the same population, we would expect it would be the same in all cases therefore it can be rewritten as
$E[\hat{\Theta}_1] = \frac{7E[X]}{7} = E[X]$.
which lead to showing that it is unbiased.
My confusing occurs when i use the Expected value on an estimator estimating $\mu$, it does not make sense to do that, (and seems incorrect to say so), since it's like finding the expected value of $\mu$, which obviously is $\mu$.
So what am I doing wrong?
Best Answer
"What am I doing wrong?" Nothing here. Your approach is correct: $\hat{\theta}_1$ is an unbiased estimator, for the reason you give (basically, it has to do with linearity of expectation). When $X_1,\dots,X_n$ are identically distributed and have finite expectation $\mu$, $$ \mathbb{E}\!\left[\frac{1}{n}\sum_{k=1}^n X_k\right] = \frac{1}{n}\sum_{k=1}^n \mathbb{E}[X_k] = \frac{1}{n}\sum_{k=1}^n \mu = \mu $$
For the same reason, $\hat{\theta}_2$ is unbiased as well: $$ \mathbb{E}\!\left[\hat{\theta}_2\right] = \frac{2\mathbb{E}[X_1] - \mathbb{E}[X_6]+\mathbb{E}[X_4]}{2} = \frac{2\mu - \mu+\mu}{2} = \mu\ . $$
Note that while both are unbiased estimators, the first one may be better, as it will have smaller variance... i.e., while both expectations are the desired value $\mu$, the second will be much less accurate (more subject to random fluctuations).
-- Attempt at addressing the larger question: assume the goal is to estimate a quantity $\theta$ of the distribution, assuming you have access to identically distributed (and, most of the time, assumed to be independent) r.v.s/samples from it, $X_1,X_2,\dots$. You basically want to come up with a nice function $f$ or sequence of functions $(f_n)$ such that either
Here, requiring an unbiased estimate you also want $\mathbb{E}[f_n(X_1,\dots, X_n)] = \theta$ (for all $n$), when the probability is over the realizations of $X_1,X_2,\dots$. Basically, your estimator is also a random variable, which depends on the $X_i$'s (being a function of them); and you can as such compute its expectation, variance, etc.
Now, for the case of estimating the expected value, things can seem a bit circular, since your estimator is basically a linear function of the $X_i$'s, and computing its expected value amounts to compute their expected value, which is what you are trying to estimate. But it does make sense nonetheless (it's not because it looks to simple to be true that it's not true); for instance, one could come up with other estimators for the expected value, say for instance $ Y = \ln \frac{e^{X_1}+e^{X_2}+\dots+e^{X_n}}{n}. $ Computing the expected value of this new random variable will not be as straightforward; yet eventually it will also boil down to using the fact that the expectation of each $X_i$ is $\mu$ (as well, maybe, as other assumptions like independence of the $X_i$'s).