The Cholesky decomposition can be used to obtain $A$ from $X = AA^{T}$ (lower triangular version) but also $B$ from $Y = B^{T}B$ (upper triangular version). The SVD can be used to do something similar to the lower triangular Cholesky decompositionas as described here; e.g. obtaining $\mathbb{V}\mathbb{D}$ from $\mathbb{C} = \mathbb{V}\mathbb{D}^{2}\mathbb{V}'$. But how can it be adapted to something similar to the upper triangular Cholesky decomposition?
Solved – How to do SVD instead of Cholesky for $L^{T}L$
cholesky decompositionsvd
Best Answer
Theory:
Let $\mathbf{z}$ be a random vector with mean zero and $\mathrm{Var}\left(\mathbf{z} \right) = I$. Let $\Sigma$ be the desired covariance matrix.
For any matrix $A$ such that $AA' = \Sigma$
$$\mathrm{Var}\left( A\mathbf{z} \right) = A\mathrm{Var}(\mathbf{z})A' = \Sigma $$
There are numerous ways to obtain such an A. Some commonly used approaches are:
Obtaining $A$ such that $AA'=\Sigma$ using Lower Triangular Cholesky
If $\Sigma$ is positive definite you can compute the Cholesky decomposition.
L = chol(Sigma,'lower')
Then $LL' = \Sigma$ and we can choose $A = L$.
Obtaining $A$ such that $AA'=\Sigma$ using Upper Triangular Cholesky
R = chol(Sigma,'upper')
Then $R'R = \Sigma$ and we can choose $A = R'$.Obtaining $A$ such that $AA'=\Sigma$ using Singular Value Decomposition
[U, S, V] = svd(Sigma)
A singular value decomposition on a matrix $\Sigma$ will yield matrices $U$, $S$, $V$ Such that $USV'=\Sigma$ and $S$ is diagonal. Furthermore, if $\Sigma$ is symmetric, we have $U = V$. Choose: $$ A = U \sqrt{S}$$ Then $AA' = U\sqrt{S}\sqrt{S}U'= USU' = USV' = \Sigma$
Let $n$ be the number of random vectors we wish to generate, let $k$ be the dimension of the vector, and let
randn
be a function which generates an $n$ by $k$ matrix representing $n\cdot k$ iid draws of variance 1 random variables.And
X
will be an $n$ by $k$ matrix withcov(X)
$\approx \Sigma$ wherecov
computes the sample covariance matrix taking each row as an observation.What if I have $B$ such that $B'B=\Sigma$?
This is not a meaningfully different case. Let $A = B'$ then $AA'=\Sigma$.