Suppose that $x_{i}|\mu,\sigma^{2} \sim \mathcal{N}(\mu,\sigma^{2})$ for $i = 1,…n$. Assume that the assigned prior distributions are $\mu$ ~ $\mathcal{N}$($\mu_{0}$, $\sigma^{2}_{0}$) and $\tau \sim Gamma(ξ_{0}, ξ_{0})$ with $\tau = \frac{1}{\sigma^{2}}$
T
he joint posterior distribution of $\mu$ and $\tau$, $p(\mu,\tau|\mathbf{x})$, where $\mathbf{x}$ is $x_{1},x_{2},…,x_{n}$, is
$$p(\mu,\tau|\mathbf{x})\propto \tau^{\frac{n}{2}+ξ_{0}-1}\exp(-\tauξ_{0})\exp\left\{-\frac{1}{2}\tau\sum_{i=1}^{n}(x_{i}-\mu)^2-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\right\}$$
and the full conditional distributions of $\mu$ and $\tau$ , $p(\mu|\tau, \mathbf{x})$ and $p(\tau|\mu, \mathbf{x})$ as
$$p(\mu|\tau, \mathbf{x}) \sim \mathcal{N}(\frac{\tau n\bar{x}\sigma_{0}^2+\mu_{0}}{n \tau\sigma_{0}^2+1} , \frac{\sigma_{0}^{2}}{n \tau\sigma_{0}^{2}+1})$$
$$p(\tau|\mu, \mathbf{x}) \sim Gamma(\frac{n}{2}+ξ_{0},\frac{\sum_{i=1}^{n}(x_{i}-\mu)^2}{2}+ξ_{0})$$
where $\bar{x} = \frac{\sum_{i=1}^{n}x_{i}}{n}.$
Now I have to derive the posterior predictive distribution $p(\tilde{x}|\mathbf{x})$, by definition $$p(\tilde{x}|\mathbf{x}) = \int\int p(\tilde{x}|\mu,\tau) p(\mu,\tau|\mathbf{x}) d\tau d\mu$$
My problem is with the exact density of $p(\mu,\tau|\mathbf{x})$. It is not a Normal-Gamma distribution since it carries the term $\exp\{-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\}$ which does not involve $\tau$. Hence I cannot proceed with the integration.
Best Answer
If $$p(\mu,\tau|\mathbf{x})\propto \tau^{\frac{n}{2}+ξ_{0}-1}\exp(-\tauξ_{0})\exp\left\{-\frac{1}{2}\tau\sum_{i=1}^{n}(x_{i}-\mu)^2-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\right\}\tag{1}$$ and (assuming $\tilde X\sim\mathcal N(\mu,\tau^{-1})$) $$p(\tilde{x}|\mu,\tau)\propto \tau^{1/2}{\sqrt{2\pi}}\exp\{-\frac{1}{2}\tau(\tilde x-\mu)^2\}$$ then $$p(\tilde{x}|\mu,\tau)p(\mu,\tau|\mathbf{x})\propto \tau^{\frac{n-1}{2}+\xi_{0}}\exp\left\{-\tau\xi_{0}-\frac{\tau(\tilde x-\mu)^2}{2}-\frac{\tau}{2}\sum_{i=1}^{n}(x_{i}-\mu)^2-\frac{(\mu-\mu_{0})^2}{2\sigma_{0}^{2}}\right\}$$ Integrating out $\tau$ gives $$p(\tilde{x}|\mu,\mathbf{x})\propto \left\{2\xi_{0}+(\tilde x-\mu)^2+\sum_{i=1}^{n}(x_{i}-\mu)^2\right\}^{-\frac{n+1}{2}-\xi_{0}}\exp\left\{-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\right\}$$ which has no closed form expression.
The issue stems from the choice of prior $$\mu\sim\mathcal N(\mu_0,\sigma_0^{})$$ since this prior is not conjugate: as you can see from (1), the posterior on $\mu,\tau)$ is not of the same form as the prior since $\mu$ and $\tau$ become dependent. A conjugate prior should involve $\tau$ as for instance $$\mu\sim\mathcal N(\mu_0,\omega\tau^{-1})$$ (as for instance detailed in our Bayesian essentials texbook, Chapter 2).