I am trying to define a 2 dimensional Gaussian in which each dimension has a different variance.
One way I thought of was just to define it as the product of 1D Gaussians, but I wasn't sure if that would hold up. Numerically plotting the product of two zero-mean, unit-variance distributions appears to give the concentric circles that one would expect in the 2D case. Furthermore, adjusting the variance of one warps the plot to concentric ellipses. However, I want to be able to express it mathematically, not numerically. Mathematically expressed, the product of two 1D Gaussians doesn't match up with the expression I'm used to seeing for a multivariable Gaussian distribution.
Multivariate Gaussian where $x \in \mathbb{R}^2$ :
$$\frac{1}{\sqrt{(2\pi)^{k}|\Sigma|}} \exp \left(-\frac{1}{2}(x – \mu)^{T}\Sigma^{-1}(x – \mu) \right)$$
Product of two 1D Gaussians:
$$\frac{1}{\sigma_{1}\sigma_{2}2\pi} \exp \left( -\frac{(x_1 – \mu_1)^2}{2\sigma_1^2} -\frac{(x_2 – \mu_2)^2}{2\sigma_2^2} \right)$$
Where am I going wrong here? Would setting $\Sigma$ in the multivariate gaussian to diag($\sigma_1$, $\sigma_2$) do the trick?
Best Answer
Let $x \sim N_2(\mu, \Sigma)$ where $\Sigma = \left [\begin{array}{cc} \sigma_1^2 & 0\\ 0 & \sigma_2^2 \end{array} \right]$. Then det($\Sigma)$ = $\sigma_1^2 \sigma_2^2$
\begin{align*} & \dfrac{1}{\sqrt{(2\pi)^2 \mathrm{det(}\Sigma)}} \exp\left \{ -\dfrac{1}{2}(x - \mu)^T \Sigma^{-1} (x - \mu) \right \}\\ = & \dfrac{1}{2\pi \sigma_1 \sigma_2} \exp \left \{-\dfrac{1}{2} \left[x_1 - \mu_1\quad x_2 - \mu_2 \right]\left [\begin{array}{cc} 1/\sigma_1^2 & 0\\ 0 & 1/\sigma_2^2 \end{array} \right] \left[ \begin{array}{c} x_1 - \mu_1\\ x_2 - \mu_2 \end{array} \right] \right \}\\ = & \dfrac{1}{2\pi \sigma_1 \sigma_2} \exp \left \{-\dfrac{1}{2} \left[\dfrac{x_1 - \mu_1}{\sigma^2_1} \quad \dfrac{x_2 - \mu_2}{\sigma^2_2} \right] \left[ \begin{array}{c} x_1 - \mu_1\\ x_2 - \mu_2 \end{array} \right] \right \}\\ = &\dfrac{1}{2\pi \sigma_1 \sigma_2} \exp \left \{ -\dfrac{(x_1 - \mu_1)^2}{2\sigma_1^2} - \dfrac{(x_2 - \mu_2)^2}{2 \sigma_2^2} \right \}. \end{align*}