[Math] Under what circumstance will a covariance matrix be positive semi-definite rather than positive definite

matricespositive-semidefiniteprobability theorystatistics

I have a covariance matrix:

$\operatorname{cov}(\mathbf{X}, \mathbf{X}) = \operatorname{E}[(\mathbf{X} – \operatorname{E}[\mathbf{X}])(\mathbf{X} – \operatorname{E}[\mathbf{X}])^T]$

According to Wikipedia, it should be a positive semi-definite matrix.

Under what circumstances will it be positive semi-definite rather than positive definite?

The reason I am asking is because I see that a common thing to do when implementing an Unscented Kalman Filter is to implement the square-root of the covariance matrix using the matlab command:

sqrt_P = gamma * chol(P_a, 'lower')

where gamma is a scaling factor and P_a is the state covariance matrix.

I understand that for chol() to work, it needs to be positive definite:

>> help chol
 CHOL   Cholesky factorization.
    CHOL(A) uses only the diagonal and upper triangle of A.
    The lower triangle is assumed to be the (complex conjugate)
    transpose of the upper triangle.  If A is positive definite, then
    R = CHOL(A) produces an upper triangular R so that R'*R = A.
    If A is not positive definite, an error message is printed.

So, what are the dangers in assuming that it isn't positive semi-definite? Will it only be semi-definite when (for example) it is the zero matrix, or when there are fully correlated states?

ADDENDUM:

In the original post, there was a reference to "if the states are fully correlated". This was rather fast and loose with the notation. There is a discussion and an answer on it here.

Best Answer

Well, in the $1 \times 1$ case, a matrix is positive semi-definite precisely when its single entry is a non-negative number, and a random variable $X$ has zero variance if and only if it is a.s. constant. (If you don't know what ‘a.s.’ means, you may ignore it throughout this discussion.) Indeed, assuming $\mathbb{E}[X] = 0$ (which is no loss of generality), we have $$\operatorname{Var} X = \int_{\Omega} X^2 \mathrm{d} \mathbb{P}$$ and since the integrand is non-negative, this is zero if and only if the integrand is a.s. zero, i.e. if and only if $X = 0$ a.s. In the case where $\mathbb{E}[X] \ne 0$, we have (by linearity) $\operatorname{Var} X = 0$ if and only if $X = \mathbb{E}[X]$ a.s.

In general, if you have a $n \times n$ symmetric matrix $V$, there is an orthogonal matrix $Q$ such that $Q V Q^{\sf T}$ is a diagonal matrix $D$, and $V$ is positive semi-definite if and only if the diagonal entries of $D$ are all non-negative. But if $V$ is the covariance matrix of $\mathbf{X}$, then $D$ is the covariance matrix of $Q \mathbf{X}$, and so $V$ is positive semi-definite but not positive definite if and only if some component of $Q \mathbf{X}$ is a.s. constant. This happens if and only if some linear combination of $\mathbf{X}$ is ‘fully correlated’, to use your phrasing.

Related Question