Consider a family of density functions $f(x|\theta)$ where the parameter space for $\theta$ is finite, that is, $\theta \in \{\theta_1, \cdots, \theta_p\}$. Assume that $\theta_p$ is such that $f(x|\theta_p) > 0$ for all $x$ on the support. Now consider the statistic $T(\mathbf{X}) = (T_1(\mathbf{X}), \cdots, T_p(\mathbf{X})) = \left(\frac{f(\mathbf{X}|\theta_1)}{f(\mathbf{X}|\theta_p)}, \cdots, \frac{f(\mathbf{X}|\theta_p)}{f(\mathbf{X}|\theta_p)} \right)$. Show that it is a minimal sufficient statistic of $\theta$.
I'm assuming I need to somehow apply the factorization theorem here to first show it is sufficient, then somehow show it is minimal. Is that the right approach? If not, how would I start?
Best Answer
Consider $f(x|\theta)$ for $\theta \in \{\theta_1, \cdots, \theta_p\}$. It can be written as \begin{align*} f(x|\theta)&=\frac{f(x|\theta)}{f(x|\theta_p)}\times f(x|\theta_p)\\ &=f(x|\theta_p)\,\exp\{\log[{f(x|\theta)}\big/{f(x|\theta_p)}]\} \end{align*} Since $\theta \in \{\theta_1, \cdots, \theta_p\}$, $\theta$ is one of the $\theta_i$'s, meaning $$\sum_{i=1}^p \mathbb{I}_{\theta=\theta_i}=\sum_{i=1}^p \mathbb{I}_{\theta_i}(\theta)=1$$ Thus $$\log[{f(x|\theta)}\big/{f(x|\theta_p)}]=\sum_{i=1}^p \mathbb{I}_{\theta_i}(\theta)\log[{f(x|\theta_i)}\big/{f(x|\theta_p)}]=\sum_{i=1}^p \mathbb{I}_{\theta_i}(\theta)\log[T_i(x)]$$and $$f(x|\theta)=\underbrace{f(x|\theta_p)}_{h(x)}\,\exp\underbrace{\left\{\sum_{i=1}^p \mathbb{I}_{\theta_i}(\theta)\log[T_i(x)]\right\}}_{R(\theta)^\text{T}S(x)}$$As a consequence, the density $f(x|\theta)$ belongs to an exponential family with natural parameter $R(\theta)$ and sufficient statistic $S(x)$. Since the components of $R(\theta)$ are linearly independent, the statistic $S(x)$ [and hence the statistic $T(x)$] are minimal. I remember seeing something like that in a paper by Don Fraser. (And this one as well.)