[Math] A non-existence example of a sufficient statistic

statistical-inferencestatistics

In parametric statistical inference, a statistic $T$ of some variable $X$ (thought of as experimental or observational data) is sufficient for the parameter $\theta$ if is captures the essential information in the data $X$ about the parameter $\theta$ (there are several equivalent definitions for this). It is implicitly understood that the random variable $X$ has a probability distribution $f(x;\theta)$, where the parameter $\theta$ is of course unknown.

Related is the concept of a minimal sufficient statistic, which captures nothing more than the essential.

  1. Is it true that any random variable $X$ as above has a sufficient statistic?
  2. Suppose $X$ has a sufficient statistic $T$. Must it also have a minimal sufficient statistic?

I'd be glad to have either a concise proof or a simple counter-example for each of these two.

Best Answer

  1. In a degenerate manner your sample $(X_1,...,X_n)$ is always a sufficient statistic from Fisher's information perspective. $T(X)$ is sufficient statistic for $\theta$ iff $\mathcal{I}_{\theta}(T(X))=\mathcal{I}_{\theta}(X_1,...,X_n)$.Thus for every parametric structure you can take the data points themselves as the sufficient statistic.
  2. No. You have a minimial sufficient statistic iff your parametric distribution $f(X;\theta)$ can be factorized into $h(X)g(\theta; T(X))$, then the $T(X)$ is the minimal sufficient statistic or can be reduced to one. Moreover, it is pretty common to fail to satisfy this condition, e.g., Weibull distribution which density function is given by $$ f(x;\theta)=\frac{\theta x^{\theta-1}}{\lambda^\theta}\exp\{-x^\theta/\lambda^\theta\}, x\ge 0, $$
    where $\theta > 0, \theta \neq 1$ is the unknown shape parameter, cannot be factorized w.r.t $\theta$.
Related Question