Is every estimator a sufficient statistic

parameter estimationstatistical-inferencestatistics

It is clear that not any sufficient statistic (s.s.) makes a good estimator (since a monotonic transform of a s.s. is still a s.s.). But is a "good" estimator of the parameter always a s.s.? If not, under what conditions is this the case?

I gather that the dimension of the s.s. may vary with the size of the data (e.g. Cauchy distribution), so perhaps we have to restrict ourselves to the exponential family of distributions for a start?

I will leave open what "good" means, e.g. it could be unbiasedness or another condition under which this result holds.

Best Answer

Sufficient statistics don't need to estimate anything. They merely pertain to data reduction; i.e., no information about the parameter that can be inferred from the sample is discarded. Every sample has, for example, the trivial sufficient statistic which is the sample itself--no data reduction is accomplished, but no information is lost, either.

The notion of sufficiency is relevant to estimation because sufficiency is often a desirable property of an estimator. However, it is not the only desirable property. Unbiasedness is also desirable, for example; however, an unbiased statistic need not be sufficient; e.g., $X_1, \ldots, X_n$ are iid random variables drawn from a parametric distribution with finite mean $\mu$ and variance $\sigma^2$; the sample mean $\bar X$ is an estimator of $\mu$ but so is $X_1$. Both are unbiased for $\mu$, but the latter is not sufficient for $\mu$.

Consistency is also a desirable property, but here again we can easily construct consistent but insufficient statistics; e.g., $(X_1 + \cdots + X_{n-1})/(n-1)$, the mean of the sample that omits the last observation, is consistent and unbiased but not sufficient because it has discarded information about $\mu$ that was present in the original sample.

So the question of what we mean by "good" when we say "good estimators" is in fact not only relevant to the question, but in a sense, it is the crux of the question you are asking. That is to say, if by "good" we mean such an estimator must not needlessly discard data, then that is how we motivate and ultimately define the notion of a sufficient statistic in the first place. If by "good" we mean some other notion, then as you can see, it is not difficult to construct estimators that fail to be sufficient yet can exhibit a number of other desirable properties, simply by omitting one observation in the sample.

Related Question