I am approximating a probability distribution over n RVs by n factors following a bivariate conditional distribution. For instance for 4 variables I could factorize p as:
$$p(x_1,x_2,x_3,x_4) = p(x_1)p(x_2|x_1)p(x_3|x_1)p(x_4|x_2)$$
The question is how to obtain for instance the marginal $p(x_1,x_3,x_4)$?
$p(x_1)$ is a normal distribution, while all other conditional are conditional bivariate normal distributions.
Best Answer
In the event $$p(x_1,x_2,x_3,x_4) = p(x_1)p(x_2|x_1)p(x_3|x_1)p(x_4|x_2)$$ only involve Normal distributions, one can without loss of generality assume that all four means are zero and all four variances are one, since otherwise, we can return to this case by a change of variables. If (and this is unclear from the question as stated) the three bivariate distributions defined by \begin{align*} p(x_1,x_2)&=p(x_1)p(x_2|x_1)\\ p(x_1,x_3)&=p(x_1)p(x_3|x_1)\\ p(x_2,x_4)&=p(x_2)p(x_4|x_2)\\ \end{align*} are bivariate Normal with correlation coefficients $\rho_{12}$, $\rho_{13}$, $\rho_{24}$, then \begin{align*} \log p(x_1,x_2)&\propto -\frac{1}{2}x_1^2-\frac{1}{2}x_2^2+\rho_{12}x_1x_2\\ \log p(x_1,x_3)&\propto-\frac{1}{2}x_1^2-\frac{1}{2}x_3^2+\rho_{13}x_1x_3\\ \log p(x_2,x_4)&\propto-\frac{1}{2}x_2^2-\frac{1}{2}x_4^2+\rho_{24}x_2x_4\\ \end{align*} (where the proportionality factor unusually means up to an additive constant). Now $$p(x_2|x_1)p(x_4|x_2)\propto \exp\left\{- \frac{x_2^2}{2(1-\rho_{12}^2)}+\frac{\rho_{12}x_1x_2}{(1-\rho_{12}^2)}-\frac{x_4^2}{2(1-\rho_{24}^2)}+\frac{\rho_{24}x_2x_4}{(1-\rho_{24}^2)} \right\}$$ (where the proportionality factor is back to its usual meaning up to a multiplicative constant). Hence \begin{align*}p(x_4|x_2) &\propto \int_{-\infty}^{\infty} \exp\left\{- \frac{x_2^2}{2(1-\rho_{12}^2)}+\frac{\rho_{12}x_1x_2}{(1-\rho_{12}^2)}-\frac{x_4^2}{2(1-\rho_{24}^2)}+\frac{\rho_{24}x_2x_4}{(1-\rho_{24}^2)} \right\}\text{d}x_2\\ &\propto \exp\left\{-\frac{x_4^2}{2(1-\rho_{24}^2)}\right\} \\ &\quad \times \int_{-\infty}^{\infty} \exp\left\{- \frac{x_2^2}{2(1-\rho_{12}^2)}+\frac{\rho_{12}x_1x_2}{(1-\rho_{12}^2)}+\frac{\rho_{24}x_2x_4}{(1-\rho_{24}^2)} \right\}\text{d}x_2\\ &\propto \exp\left\{-\frac{x_4^2}{2(1-\rho_{24}^2)}\right\} \\ &\quad \times \int_{-\infty}^{\infty} \exp\frac{-1}{2(1-\rho_{12}^2)}\left\{x_2^2-2\rho_{12}x_1x_2-\frac{2(1-\rho_{12}^2)\rho_{24}x_2x_4}{(1-\rho_{24}^2)} \right\}\text{d}x_2\\ &\propto \exp\left\{-\frac{x_4^2}{2(1-\rho_{24}^2)}\right\} \\ &\quad \times \int_{-\infty}^{\infty} \exp\frac{-1}{2(1-\rho_{12}^2)}\left\{\left(x_2-\rho_{12}x_1-\frac{(1-\rho_{12}^2)\rho_{24}x_4}{(1-\rho_{24}^2)} \right)^2\right\}\text{d}x_2\\ &\quad\times \exp\frac{+ 1}{2(1-\rho_{12}^2)}\left\{\left( \rho_{12}x_1 + \frac{(1-\rho_{12}^2)\rho_{24}x_4}{(1-\rho_{24}^2)} \right)^2\right\}\\ &\propto \exp\left\{-\frac{x_4^2}{2(1-\rho_{24}^2)}\right\} \\ &\quad \times \int_{-\infty}^{\infty} \exp\frac{-1}{2(1-\rho_{12}^2)}\left\{\left(x_2-\rho_{12}x_1-\frac{(1-\rho_{12}^2)\rho_{24}x_4}{(1-\rho_{24}^2)} \right)^2\right\}\text{d}x_2\\ &\quad \times \exp\frac{+ 1}{2(1-\rho_{12}^2)}\left\{\left( \rho_{12}x_1 + \frac{(1-\rho_{12}^2)\rho_{24}x_4}{(1-\rho_{24}^2)} \right)^2\right\}\\ &\propto \exp\left\{-\frac{x_4^2}{2(1-\rho_{24}^2)}\right\} \\ &\quad \times \exp\frac{+ 1}{2(1-\rho_{12}^2)}\left\{\left( \rho_{12}x_1 + \frac{(1-\rho_{12}^2)\rho_{24}x_4}{(1-\rho_{24}^2)} \right)^2\right\} \end{align*}