I have been researching the meaning of positive semi-definite property of correlation or covariance matrices.
I am looking for any information on
- Definition of positive semi-definiteness;
- Its important properties, practical implications;
- The consequence of having negative determinant, impact on multivariate analysis or simulation results etc.
Best Answer
The variance of a weighted sum $\sum_i a_i X_i$ of random variables must be nonnegative for all choices of real numbers $a_i$. Since the variance can be expressed as $$\operatorname{var}\left(\sum_i a_i X_i\right) = \sum_i \sum_j a_ia_j \operatorname{cov}(X_i,X_j) = \sum_i \sum_j a_ia_j \Sigma_{i,j},$$ we have that the covariance matrix $\Sigma = [\Sigma_{i,j}]$ must be positive semidefinite (which is sometimes called nonnegative definite). Recall that a matrix $C$ is called positive semidefinite if and only if $$\sum_i \sum_j a_ia_j C_{i,j} \geq 0 \;\; \forall a_i, a_j \in \mathbb R.$$