[Math] What are the restrictions of a covariance matrix

covariance

This question is not a duplication of this question this question.

I want to generate a random covariance matrix so at first, I should know what a covariance matrix should be like.

Surely, we know that the covariance matrix of a series events must be symmetric and positive semi-definite. On the other hand,
$$\text{Cov}(X,X)\text{Cov}(Y,Y)\geq\text{Cov}(X,Y)^2$$
and
$$\text{Cov}(X,X)\geq 0 $$
Then I suppose any matrix meets these restrictions can be a matrix. Is that right?

Further more, I decide to construct a symmetric matrix and check its positive semi-definiteness rather than generate a positive semi-definite matrix first. So I use the rest of these restrictions above, to construct a matrix. In my opinion, it is ok, for the covariance of two event is a random number, and there is not relation between the other covariance, which means the covariance can be random if it meets the rest of these restrictions above, and it will be positive semi-definite naturally.

BUT, it is not true. for example:
$$A = \begin{bmatrix}
7 & 5 & 3 \\
5 & 9 & 8.5 \\
3 & 8.5 & 8.8
\end{bmatrix}
$$
is not a positive semi-definite matrix.

so, what on earth the covariance matrix should be?

EDIT:
Another probability is that a covariance maybe a high-dimensional 'ellipsoid', so it should be able to represent by
$$
C = U^T\Lambda U
$$
which $U$ is a Hermitian matrix, and $\Lambda$ is a positive semi-definite diagonal matrix.

So, if a matrix is diagonalizable, it can be a covariance matrix, and vice versa.

Best Answer

The covariance between two random variables is not a random number, but a deterministic value. But this isn't important to your question.

A matrix is a covariance matrix iff it is positive semidefinite. There are no other restrictions one must consider. To see this, observe that every positive semidefinite matrix $A$ has a square root $A^{1/2}$ [this follows from the principal axis theorem]. Now let $X$ be any random vector with identity cocariance matrix. Then the covariance matrix of $A^{1/2}X$ is $A$.

If you want to create a covariance matrix, you should use your second approach. Creating a random orthogonal matrix is easy and creating a random diagonal matrix is trivial.

Related Question