Let $ x_i $ be iid observations in a sample from a uniform distribution over $ \left[ 0, \theta \right] $. Now I need to estimate $ \theta $ based on $N$ observations and I want the estimator to be unbiased.
I thought about simple estimator $ \hat{\theta} = \max \left( x_i \right) $.
Based on simulation it is not biased, yet I couldn't show it analytically.
Could anyone, please, show it is unbiased?
BTW, I could easily find another, easy to prove, unbiased estimator, $ \hat{\theta} = 2 \mathrm{mean} \left( {x}_{i} \right) $
Best Answer
Of course $\hat\theta=\max\{x_i\}$ is biased, simulations or not, since $\hat\theta<\theta$ with full probability hence one can be sure that, for every $\theta$, $E_\theta(\hat\theta)<\theta$.
One usually rather considers $\hat\theta_N=\frac{N+1}N\cdot\max\{x_i\}$, then $E_\theta(\hat\theta_N)=\theta$ for every $\theta$.
To show that $\hat\theta_N$ is unbiased, one must compute the expectation of $M_N=\max\{x_i\}$, and the simplest way to do that might be to note that for every $x$ in $(0,\theta)$, $P_\theta(M_N\le x)=(x/\theta)^N$, hence $$ E_\theta(M_N)=\int\limits_0^{\theta}P_\theta(M_N\ge x)\text{d}x=\int\limits_0^{\theta}(1-(x/\theta)^N)\text{d}x=\theta-\theta/(N+1)=\theta N/(N+1). $$
Edit (Below is a remark due to @cardinal, which completes nicely this post.)
It may be worth noting that the maximum $M_N=\max\limits_{1\le k\le N}X_k$ of an i.i.d. Uniform$(0,θ)$ random sample is a sufficient statistic for $θ$ and that it is one of the few statistics for distributions outside the exponential family which is also complete.
An immediate consequence is that $\hat\theta_N=(N+1)M_N/N$ is the uniformly minimum variance unbiased estimator (UMVUE) for $θ$, that is, that any other unbiased estimator for $θ$ is a worse estimator in the $L^2$ sense.