Solved – Finding MLE and MSE of $\theta$ where $f_X(x\mid\theta)=\theta x^{−2} I_{x\geq\theta}(x)$

density functionestimationinferencemaximum likelihoodself-study

Consider i.i.d random variables $X_1$, $X_2$, . . . , $X_n$ having pdf

$$f_X(x\mid\theta) = \begin{cases} \theta x^{−2} & x\geq\theta \\ 0 &
x\lt\theta \end{cases}$$

where $\theta \gt 0$ and $n\geq 3$

(a) Give the likelihood function, expressing it clearly as a function
of $\theta$

(b) Give the MSE of the MLE of $\theta$

My Attempt:

(a) $$L(\theta\mid\vec{x}) = \begin{cases} \theta^n\left(\prod_{i=1}^n x_i\right)^{−2} & x_{(1)}\geq\theta \\ 0 &
x_{(1)}\lt\theta \end{cases}$$

(b) Clearly the MLE of $\theta$ is $X_{(1)}$. We have

&=\mathsf P(\text{min}{\{X_1,…,X_n}\}\leq x)\\\\
&=1-\mathsf P(\text{min}{\{X_1,…,X_n}\}\gt x)\\\\



It follows that

\mathsf E\left(X_{(1)}^2\right)


\mathsf E\left(X_{(1)}\right)



We also have

&=\left(\mathsf E\left(\hat{\theta}\right)-\theta\right)^2\\\\

Finally the MSE is given by


Are these valid solutions?

Best Answer

This question is now old enough to give a full succinct solution confirming your calculations. Using standard notation for order statistics, the likelihood function here is:

$$\begin{aligned} L_\mathbf{x}(\theta) &= \prod_{i=1}^n f_X(x_i|\theta) \\[6pt] &= \prod_{i=1}^n \frac{\theta}{x_i^2} \cdot \mathbb{I}(x_i \geqslant) \\[6pt] &\propto \prod_{i=1}^n \theta \cdot \mathbb{I}(x_i \geqslant \theta) \\[12pt] &= \theta^n \cdot \mathbb{I}(0 < \theta \leqslant x_{(1)}). \\[6pt] \end{aligned}$$

This function is strictly increasing over the range $0 < \theta \leqslant x_{(1)}$ so the MLE is:

$$\hat{\theta} = x_{(1)}.$$

Mean-squared-error of MLE: Rather than deriving the distribution of the estimator, it is quicker in this case to derive the distribution of the estimation error. Define the estimation error as $T \equiv \hat{\theta} - \theta$ and note that it has distribution function:

$$\begin{aligned} F_T(t) \equiv \mathbb{P}(\hat{\theta} - \theta \leqslant t) &= 1-\mathbb{P}(\hat{\theta} > \theta + t) \\[6pt] &= 1-\prod_{i=1}^n \mathbb{P}(X_i > \theta + t) \\[6pt] &= 1-(1-F_X(\theta + t))^n \\[6pt] &= \begin{cases} 0 & & \text{for } t < 0, \\[6pt] 1 - \Big( \frac{\theta}{\theta + t} \Big)^n & & \text{for } t \geqslant 0. \\[6pt] \end{cases} \end{aligned}$$

Thus, the density has support over $t \geqslant 0$, where we have:

$$\begin{aligned} f_T(t) \equiv \frac{d F_T}{dt}(t) &= - n \Big( - \frac{\theta}{(\theta + t)^2} \Big) \Big( \frac{\theta}{\theta + t} \Big)^{n-1} \\[6pt] &= \frac{n \theta^n}{(\theta + t)^{n+1}}. \\[6pt] \end{aligned}$$

Assuming that $n>2$, the mean-squared error of the estimator is therefore given by:

$$\begin{aligned} \text{MSE}(\hat{\theta}) = \mathbb{E}(T^2) &= \int \limits_0^\infty t^2 \frac{n \theta^n}{(\theta + t)^{n+1}} \ dt \\[6pt] &= n \theta^n \int \limits_0^\infty \frac{t^2}{(\theta + t)^{n+1}} \ dt \\[6pt] &= n \theta^n \int \limits_\theta^\infty \frac{(r-\theta)^2}{r^{n+1}} \ dr \\[6pt] &= n \theta^n \int \limits_\theta^\infty \Big[ r^{-(n-1)} - 2 \theta r^{-n} + \theta^2 r^{-(n+1)} \Big] \ dr \\[6pt] &= n \theta^n \Bigg[ -\frac{r^{-(n-2)}}{n-2} + \frac{2 \theta r^{-(n-1)}}{n-1} - \frac{\theta^2 r^{-n}}{n} \Bigg]_{r = \theta}^{r \rightarrow \infty} \\[6pt] &= n \theta^n \Bigg[ \frac{\theta^{-(n-2)}}{n-2} - \frac{2 \theta^{-(n-2)}}{n-1} + \frac{\theta^{-(n-2)}}{n} \Bigg] \\[6pt] &= n \theta^2 \Bigg[ \frac{1}{n-2} - \frac{2}{n-1} + \frac{1}{n} \Bigg] \\[6pt] &= \theta^2 \cdot \frac{n(n-1) - 2n(n-2) + (n-1)(n-2)}{(n-1)(n-2)} \\[6pt] &= \theta^2 \cdot \frac{n^2 - n - 2n^2 + 4n + n^2 - 3n + 2}{(n-1)(n-2)} \\[6pt] &= \frac{2\theta^2}{(n-1)(n-2)}. \\[6pt] \end{aligned}$$