I am currently studying sufficiency statistics. My notes say the following:
A statistic $T(\mathbf{Y})$ is sufficient for $\theta$ if, and only if, for all $\theta \in \Theta$,
$$L(\theta; \mathbf{y}) = g(T(\mathbf{y}), \theta) \cdot h(\mathbf{y}),$$
where the function $g(\cdot)$ depends on $\theta$ and the statistic $T(\mathbf{Y})$, while the function $h(\cdot)$ does not contain $\theta$.Sufficient statistics are not unique:
Any one-to-one transformation of a sufficient statistic is again a sufficient statistic.Sufficiency depends on the model:
- Let $Y_1, \dots, Y_n$ be a sample from $N(\mu, \sigma^2)$, where $\sigma^2 > 0$ is known. The only unknown parameter is $\mu = E[Y]$.
$T(\mathbf{Y}) = \sum_{i = 1}^n Y_i$ or $T(\mathbf{Y}) = \bar{Y}$ are sufficient statistics for $\mu$.- Let $Y_1, \dots, Y_n$ be a sample from a $\text{Uniform}[ \mu – 1, \mu + 1]$ distribution. The only unknown parameter is $\mu = E[Y]$.
In this case, $T(\mathbf{Y}) = \sum_{i = 1}^n Y_i$ or $T(\mathbf{Y}) = \bar{Y}$ are not sufficient statistics for $\mu$.
I don't understand why $T(\mathbf{Y}) = \sum_{i = 1}^n Y_i$ or $T(\mathbf{Y}) = \bar{Y}$ are sufficient statistics for $\mu$ for the normally distributed case but $T(\mathbf{Y}) = \sum_{i = 1}^n Y_i$ or $T(\mathbf{Y}) = \bar{Y}$ are not sufficient statistics for $\mu$ in the uniformly distributed case. I know that the unique characteristic of the uniform distribution is that its density is the same everywhere in the distribution, unlike the normal distribution, so I strongly suspect that this has something to do with it; although, as I said, I'm not sure precisely why.
An accompanying example for the uniformly distributed case is as follows:
Example
Let $Y_1, \dots, Y_n$ be an i.i.d. $U[\mu – 1, \mu + 1]$. It has the density
$$f_\mu (y) = \begin{cases} 1/2 & \text{if} \ \mu – 1 \le y \le \mu + 1 \\ 0 & \text{otherwise}, \end{cases}$$
where $\mu \in \Theta = \mathbb{R} = (-\infty, \infty)$. The likelihood is given by
$$\begin{align} L(\mu; \mathbf{y}) = \prod_{i = 1}^n f_{\mu} (y_i) &= \begin{cases} 1/2^n & \text{if} \ \mu – 1 \le y \le \mu + 1 \\ 0 & \text{otherwise} \end{cases} \\ &= \begin{cases} 1/2^n & \text{if} \ \mu – 1 \le y_{(1)} \le \dots \le y_{(n)} \le \mu + 1 \\ 0 & \text{otherwise} \end{cases} \\ &= \begin{cases} 1/2^n & \text{if} \ y_{(n)} – 1 \le \mu \le y_{(1)} + 1 \\ 0 & \text{otherwise} \end{cases} \end{align}$$
here $(y_{(1)} \le \dots \le y_{(n)})$ is the order statistic of $(y_1, \dots, y_n)$.
The only part of this example that is unclear to me is the last case:
$$\begin{cases} 1/2^n & \text{if} \ y_{(n)} – 1 \le \mu \le y_{(1)} + 1 \\ 0 & \text{otherwise} \end{cases}$$
Specifically, I don't understand where $\ y_{(n)} – 1 \le \mu \le y_{(1)} + 1$ came from; the equivalence of the algebra to the two cases that came before it are not clear to me.
I would greatly appreciate it if people would please take the time to clarify this.
Best Answer
Normal distribution:
Let's compute the likelihood function in order to use the Factorization theorem
\begin{align} L(\mu; y) & = \prod_{i=1}^n \frac{1}{\sqrt{2\pi\sigma^2}} \exp \left (-\frac{(y_i-\mu)^2}{2\sigma^2} \right ) \\ &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left ( -\sum_{i=1}^n \frac{(y_i-\mu)^2}{2\sigma^2} \right ) \\ & = (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left (-\sum_{i=1}^n \frac{ \left ( \left (y_i-\overline{y} \right ) - \left (\mu-\overline{y} \right ) \right )^2}{2\sigma^2} \right ) \\ & = (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \left(\sum_{i=1}^n(y_i-\overline{y})^2 + \sum_{i=1}^n(\mu-\overline{x})^2 -2\sum_{i=1}^n(y_i-\overline{y})(\mu-\overline{y})\right) \right) \\ &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \left(\sum_{i=1}^n(y_i-\overline{y})^2 + \sum_{i=1}^n(\mu-\overline{x})^2 -2(\mu-\overline{y})\sum_{i=1}^n(y_i-\overline{y})\right) \right)\\ &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \left (\sum_{i=1}^n(y_i-\overline{y})^2 + n(\mu-\overline{y})^2 \right ) \right ) \\ &= (2\pi\sigma^2)^{-\frac{n}{2}} \exp \left( -{1\over2\sigma^2} \sum_{i=1}^n (y_i-\overline{y})^2 \right ) \exp \left (-\frac{n}{2\sigma^2} (\mu-\overline{y})^2 \right ) \end{align}
We can see that only the final term involves $\mu$ and $\bar{y}$ is the only statistics involved there. Hence for the normal distribution $\bar{y}$ is a sufficient statistics.
Uniform distribution:
However, for uniform distribution, as your note has illustrated,
$$L(\mu; y) = \frac1{2^n}\mathbb{1}_{\{y^{(n)}-1 \le \mu \le y^{(1)}+1\}}$$
We define $A_x = (x^{(n)}-1, x^{(1)}+1)$.
\begin{align} \frac{L(\mu; x)}{L(\mu; y)} = \begin{cases} 0 &, \mu \notin A_x, \mu \in A_y \\ 1 & ,\mu \in A_x, \mu \in A_y \\ \infty & ,\mu \in A_x, \mu \notin A_y \end{cases} \end{align}
This depends on $\mu$ unless $A_x = A_y$.
Hence $(y^{(1)}, y^{(n)})$ is a minimal sufficient.
Note that $(y^{(1)}, y^{(n)})$ is not determined by the average. Hence the sample average is not a sufficient statistics.