Assume that $Y_{i} \sim N\left(\theta x_{i}^{2}, \sigma^{2}\right)$ for $i = 1,\ldots,n$ independently. Consider a Bayesian model with a known $\sigma = 1$ and with a prior distribution $\theta \sim N\left(0, \sigma_{0}^{2}\right)$ for some fixed value $\sigma_0 > 0$.
Show that the posterior density of $\theta$ is $$\theta \mid \mathbf{y} \sim N\left(\tilde{\theta},\left(\sum_{i=1}^{n} x_{i}^{4}+\sigma_{0}^{-2}\right)^{-1}\right)$$
and find $\tilde{\theta}$
By using the fact
$p(\theta\mid \textbf{y}) \propto {f (\textbf{y}|\theta)p(\theta)}$, how do i go about deriving the desired posterior density?
Thank you
Best Answer
Ok Let's start!
$$p(\theta|\mathbf{y}) \propto Exp\Bigg\{-\frac{\theta^2}{2\sigma_0^2}\Bigg\}Exp\Bigg\{-\frac{1}{2}\Sigma_i(y_i-\theta x_i^2)^2\Bigg\}$$
Now let's expand the exponent throwing away any element not depending on $\theta$ (it will be part of the normalization constant)
$$\propto Exp\Bigg\{-\frac{\theta^2}{2\sigma_0^2}+\frac{2\theta\Sigma_i y_ix_i^2}{2}-\frac{\theta^2\Sigma_i x_i^4}{2}\Bigg\}$$
$$\propto Exp\Bigg\{-\frac{1}{2}\Big[\theta^2\Big(\sigma_0^{-2}+\Sigma_i x_i^4\Big)-2\theta\cdot \Sigma_i y_ix_i^2\Big]\Bigg\}$$
$$\propto Exp\Bigg\{-\frac{1}{2\Big(\sigma_0^{-2}+\Sigma_i x_i^4\Big)^{-1}}\Bigg[\theta^2-2\theta\cdot\underbrace{\frac{\Sigma_iy_i x_i^2}{\sigma_0^{-2}+\Sigma_i x_i^4}}_{=\tilde{\theta}}+?\Big]\Bigg\}$$
As you can see in the exponent there is a missing square. To complete the gaussian is enough to sum and subtract the missing element, $\pm(\tilde{\theta})^2$, and multiply all by the normalization constant (but it is not necessary to answer to your question).
The mean you are requesting to shoe, in fact, is exactly the expression of $\tilde{\theta}$ I underbraced.
The variance they ask to show is evidently written...
I hope this help.