[Math] Normal distribution with standard deviation = I

normal distributionstandard deviation

Suppose a vector $\epsilon \in \mathbb R^d$ is a random vector drawn from the isotropic normal distribution:

$\epsilon$ ~ $\mathcal N (0, I)$

[As in Eq. 1.34 here.]

I suppose I is the identity matrix; I don't understand what it means to draw a vector out of a normal distribution with $\mu = 0$ and and $\sigma = I$ – i.e., when the standard deviation is a matrix. Any explanations?

Best Answer

In this particular case, it means that you draw $d$ times a $N(0,1)$ (real) random variable, and these random variables are independent.

It means that $\varepsilon=(\varepsilon_1,\dots,\varepsilon_d)$, the $\varepsilon_i$ are independent, and are normally distributed, with variance $1$ and mean $0$.

If you don't have the identity matrix but another symmetric positive $d\times d$ matrix $\Sigma_{ij}$, then you'd have $\Sigma_{ij}=Cov(\varepsilon_i,\varepsilon_j)$. You can also have a non-zero mean vector $\mu=(\mu_1,\dots,\mu_d)$, with $\mu_i=E(\varepsilon_i)$. In this case you'd write $\epsilon\sim N(\mu,\Sigma)$.