inferenceprobabilityself-studysufficient-statistics
If $X_1$ and $X_2$ are random samples from an exponential$(\frac{1}{\theta})$ distribution, how can I show that $T = X_1 + 2X_2$ is not sufficient for $\theta$ (mean of the distribution)?
I know that a statistic can be considered sufficient if I can write the probability function as:
$p(X) = h(X)g(\theta,T(X))$
Which is the factorization theorem. But how can I show that it is not possible to write it this way? Or is there any other way?
Lemma The minimal sufficient statistic $\left(X_{(1)},\sum_{i=2}^n \{X_{(i)}-X_{(1)}\}\right)$ is not complete.
Proof. The joint distribution of $$\left(X_{(1)},\sum_{i=2}^n \{X_{(i)}-X_{(1)}\}\right)$$ is the product of an Exponential $\mathcal E(n/\theta^2)$ translated by $\theta$ and of a $\mathcal Ga(n-1,1/\theta^2)$ [the proof follows from Sukhatme's Theorem, 1937, recalled in Devroye's simulation bible (1986, p.211)]. This means that $X_{(1)}$ can be represented as $$X_{(1)}=\frac{\theta^2}{n}\varepsilon+\theta\qquad\varepsilon\sim\mathcal E(1)$$ that $Y$ is scaled by $\theta^2$ since $$Y=\sum_{i=2}^n \{X_{(i)}-X_{(1)}\}=\theta^2 \eta\qquad\eta\sim\mathcal Ga(n-1,1)$$ and that $$\mathbb E_\theta\left[ Y^\frac{1}{2}\right]=\theta \frac{\Gamma(n-1/2)}{\Gamma(n-1)}$$ Therefore, $$\mathbb E_\theta\left[X_{(1)}-\frac{\Gamma(n-1)}{\Gamma(n-1/2)}Y^\frac{1}{2}\right]=\frac{\theta^2}{n}$$ eliminates the location part in $X_{(1)}$ and suggests dividing by $Y$ to remove the scale part: since $$\mathbb E_\theta\left[ Y^\frac{-1}{2}\right]=\theta^{-1} \frac{\Gamma(n-3/2)}{\Gamma(n-1)}\qquad \mathbb E_\theta\left[ Y^{-1}\right]=\theta^{-2} \frac{\Gamma(n-2)}{\Gamma(n-1)}$$ we have (for an arbitrary $\gamma)$ that $$\mathbb E_\theta\left[\frac{X_{(1)}-\gamma Y^\frac{1}{2}}{Y}\right]=\frac{\Gamma(n-2)}{n\Gamma(n-1)}+\frac{\theta^{-1}\Gamma(n-2)}{\Gamma(n-1)}- \frac{\gamma \theta^{-1}\Gamma(n-3/2)}{\Gamma(n-1)} $$ Setting $$\gamma=\frac{\Gamma(n-2)}{\Gamma(n-3/2)}$$ leads to $$\mathbb E_\theta\left[\frac{X_{(1)}-\gamma Y^\frac{1}{2}}{Y}\right]=\frac{\Gamma(n-2)}{n\Gamma(n-1)}$$ which is constant in $\theta$. Therefore this concludes the proof.
As pointed out by Sextus Empiricus, this is not the only transform of the sufficient statistic with constant expectation. His proposal $$\mathbb E_\theta\left[ X - \frac{1}{n(n-1)}Y- \frac{\Gamma(n-1)}{\Gamma(n-1/2)}Y^{1/2}\right] = 0$$is an alternative.
I hAVE tried to find the conditional distribution for two cases only...you should try to find the conditional distribution for other two cases.It will clear your doubt.
Best Answer
The idea is to find a way to change the data that does not alter $T$ but does alter the likelihood $P$ in an important way: namely, by an amount that truly depends on the parameter $\theta$. That would show that $T$ does not tell us all we need to know about $\theta$, insofar as it might be revealed through $P$. One elementary (yet fully rigorous) approach is sketched below.
Because the PDF of the underlying probability law is
$$f_\theta(x) = \frac{1}{\theta}\exp\left(x/\theta\right),$$
the assumption that $X_1$ and $X_2$ are independent implies
$$P(\mathbf{x}) = f_\theta(x_1)f_\theta(x_2) = \frac{1}{\theta^2}\exp((x_1+x_2)/\theta).$$
Suppose (in order to derive a contradiction) that this likelihood can be factored as
$$P(\mathbf{x}) = h(\mathbf{x})g(\theta, T(\mathbf{x})) = h(x_1,x_2)g(\theta, x_1+2x_2).$$
Taking logarithms will simplify things:
$$-2\log\theta + \frac{x_1+x_2}{\theta}=\log P(\mathbf{x}) = \log h(x_1,x_2) + \log g(\theta, x_1+2x_2).\tag{1}$$
Since both $x_1$ and $x_2$ are almost surely positive, for sufficiently small but nonzero $\epsilon \lt x_2$ both $x_1 + 2\epsilon$ and $x_2-\epsilon$ will still be positive, so it makes sense to plug them into both sides. Notice how this combination of changes in the $x_i$ was chosen to leave $g$ unchanged, because $$T(x_1,x_2) = T(x_1+2\epsilon, x_2-\epsilon).$$
At this juncture, compute the change in the right hand side of $(1)$ and the change in the left hand side when $(x_1,x_2)$ is changed to $(x_1+2\epsilon, x_2-\epsilon)$. (This requires only simple algebra.) Observe that the change in the RHS depends only on $x_1, x_2,$ and $\epsilon$ (by construction), but that the change in the LHS depends ineluctably on $\theta$ (because $\epsilon$ is nonzero). Draw your conclusions from this contradiction.