[Math] Hypothesis test between two normal distributions

estimation-theoryprobability distributions

Let $T_1,T_2,\ldots ,T_ n$ be i.i.d. observations, each drawn from a common normal distribution with mean zero. With probability $1/2$ this normal distribution has variance $1$, and with probability $1/2$ it has variance $4$. Based on the observed values $t_1,t_2,\ldots ,t_ n$, we use the MAP rule to decide whether the normal distribution from which they were drawn has variance $1$ or variance $4$. The MAP rule decides that the underlying normal distribution has variance $1$ if and only if$$\left| c_1 \sum _{i=1}^{n} t_ i^2 + c_2 \sum _{i=1}^{n} t_ i \right| < 1.$$

Find the values of $c_1\geq 0$ and $c_2\geq 0$ such that this is true. Express your answer in terms of $n$.

I'm finding it conceptually difficult to understand the nature of the posterior distribution of the variance of a normal distribution, given the observed $t_ i$.I'd greatly appreciate it if someone would please provide some helpful hint. Thanks.

Best Answer

Using Bayes' Rule, the posterior distribution of the variance is given by the product of the likelihood and the prior distribution of the variance divided by the marginal likelihood $$p(\sigma^2|t_1,t_2,\cdots,t_n)=\frac{p(t_1,t_2,\cdots,t_n|\sigma^2)p(\sigma^2)}{ p(t_1,t_2,\cdots,t_n)}=\frac{p(t_1,t_2,\cdots,t_n|\sigma^2)p(\sigma^2)}{\int p(t_1,t_2,\cdots,t_n|\sigma^2)p(\sigma^2)d(\sigma^2)} $$ As the observed values $t_i$ are i.i.d. (with zero mean) the likelihood function $p(t_1,t_2,...,t_n|\sigma^2)$ is given by $$p(t_1,t_2,...,t_n|\sigma^2)=\prod_{i=1}^n\frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{t_i^2}{2\sigma^2}\right)=\frac{1}{\sigma^n(2\pi)^{n/2}}\exp\left(-\frac{1}{2\sigma^2}\sum_{i=1}^nt_i^2\right)$$ while the prior on the variance is simply the following probability mass function $$p(\sigma^2=1)=\frac{1}{2},\ p(\sigma^2=4)=\frac{1}{2}$$ Given the discrete nature of the prior of the variance, the marginal likelihood simplifies to $$\begin{align}p(t_1,t_2,\cdots,t_n)&=p(t_1,t_2,\cdots,t_n|\sigma^2=1)p(\sigma^2=1)+p(t_1,t_2,\cdots,t_n|\sigma^2=4)p(\sigma^2=4)\\&=\frac{1}{2}\left[\frac{1}{(2\pi)^{n/2}}\exp\left(-\frac{1}{2}\sum_{i=1}^nt_i^2\right)+\frac{1}{2^n(2\pi)^{n/2}}\exp\left(-\frac{1}{8}\sum_{i=1}^nt_i^2\right)\right]\end{align}$$ In order to use the MAP rule to determine whether the samples were drawn from a distribution where the variance was $1$, we need to satisfy the following inequality (have a look at Equations $3.4$ and $3.5$ in here):- $$\frac{p(t_1,t_2\cdots,t_n|\sigma^2=4)p(\sigma^2=4)}{p(t_1,t_2,\cdots,t_n)}<\frac{p(t_1,t_2\cdots,t_n|\sigma^2=1)p(\sigma^2=1)}{p(t_1,t_2,\cdots,t_n)}$$ which (given the prior of the variance) simplifies to $$p(t_1,t_2\cdots,t_n|\sigma^2=4)<p(t_1,t_2\cdots,t_n|\sigma^2=1)$$ which is the condition that $$\frac{1}{2^n(2\pi)^{n/2}}\exp\left(-\frac{1}{8}\sum_{i=1}^nt_i^2\right)<\frac{1}{(2\pi)^{n/2}}\exp\left(-\frac{1}{2}\sum_{i=1}^nt_i^2\right)$$ Taking the logarithm of both sides and simplifying results in the following inequality $$\frac{3}{8n\log 2}\sum_{i=1}^nt_i^2<1$$ Thus, we have $c_1=\frac{3}{8n\log 2}$ and $c_2=0$.

Related Question