Derivative of Quadratic Form of Cholesky Factor

linear algebramatrix decompositionmatrix-calculuspositive definitesymmetric matrices

Let $\mathbf{\Sigma}$ be a symmetric positive definite matrix and let $\mathbf{L}$ be its lower Cholesky factor. Furthermore, let $\mathbf{D}$ be a diagonal matrix.

What is
$$
\begin{align}
\frac{\partial \text{vech} (\mathbf{L} \mathbf{D} \mathbf{L}^\top)}{\partial \text{vech}(\mathbf{\Sigma})^\top} \quad ?
\end{align}
$$

Probably this and this question is relevant.

Best Answer

We can use the chain rule,

$$ \begin{align} \frac{\partial \text{vech} (\mathbf{L} \mathbf{D} \mathbf{L}^\top)}{\partial \text{vech}(\mathbf{\Sigma})^\top} &= \frac{\partial \text{vech} (\mathbf{L} \mathbf{D} \mathbf{L}^\top)}{\partial \text{vech}(\mathbf{L})^\top} \frac{\partial \text{vech} (\mathbf{L})}{\partial \text{vech}(\mathbf{\Sigma})^\top} \\ &= \mathbf{G}^{+} \left( \mathbf{L} \mathbf{D} \otimes \mathbf{I} \right) \mathbf{F}^{\top} \left( \mathbf{G}^{+} \left(\mathbf{L} \otimes \mathbf{I}\right) \mathbf{F}^{\top} \right)^{-1}. \end{align} $$

To see this, let $\mathbf{G}$ be the duplication matrix, $\mathbf{G}^{+}$ its Moore-Penrose inverse, $\mathbf{F}$ the canonical elimination matrix (which consists of 0's and 1's only) and $\mathbf{K}$ the commutation matrix. Then $$ \begin{align} \mathrm{d} \mathrm{vech}\left( \mathbf{L} \mathbf{D} \mathbf{L}^{\top} \right) &= \mathbf{G}^{+} \, \, \mathrm{d} \mathrm{vec} \left( \mathbf{L} \mathbf{D} \mathbf{L}^{\top} \right) \\ &= \mathbf{G}^{+} \left( \mathrm{vec}\left( \mathrm{d} \mathbf{L} \mathbf{D} \mathbf{L}^{\top} \right) + \mathrm{vec}\left( \mathbf{L} \mathbf{D} \mathrm{d} \mathbf{L}^{\top} \right) \right) \\ &= \mathbf{G}^{+}\left(\mathbf{I} + \mathbf{K}_{pp}\right) \mathrm{vec}\left( \mathrm{d} \mathbf{L} \mathbf{D} \mathbf{L}^{\top} \right) \\ &= \mathbf{G}^{+} \left(\mathbf{I} + \mathbf{K}_{pp}\right) \left( \mathbf{L} \mathbf{D} \otimes \mathbf{I} \right) \mathrm{vec}\left( \mathrm{d} \mathbf{L} \right) \\ &= 2 \mathbf{G}^{+} \mathbf{G} \mathbf{G}^{+} \left( \mathbf{L} \mathbf{D} \otimes \mathbf{I} \right) \mathbf{F}^{\top} \mathrm{d} \mathrm{vech}\left( \mathbf{L} \right) \\ &= 2 \mathbf{G}^{+} \left( \mathbf{L} \mathbf{D} \otimes \mathbf{I} \right) \mathbf{F}^{\top} \mathrm{d} \mathrm{vech}\left( \mathbf{L} \right), \end{align} $$ and $$ \begin{align} \mathrm{d} \mathrm{vech}\left( \mathbf{\Sigma} \right) &= \mathbf{G}^{+} \mathrm{d} \mathrm{vec}\left( \mathbf{L} \mathbf{L}^{\top} \right) \\ &= \mathbf{G}^{+} \mathrm{vec}\left( \mathrm{d} \mathbf{L} \mathbf{L}^{\top} \right) + \mathbf{G}^{+} \mathrm{vec}\left( \mathbf{L} \mathrm{d} \mathbf{L}^{\top} \right) \\ &= \mathbf{G}^{+} \left(\mathbf{I} + \mathbf{K}_{pp}\right) \mathrm{vec}\left( \mathrm{d} \mathbf{L} \mathbf{L}^{\top} \right) \\ &= \mathbf{G}^{+} \mathbf{G} \mathbf{G}^{+} \mathrm{vec}\left( \mathrm{d} \mathbf{L} \mathbf{L}^{\top} \right) \\ &= 2 \mathbf{G}^{+} \left(\mathbf{L} \otimes \mathbf{I}\right) \mathrm{vec}\left( \mathrm{d} \mathbf{L} \right) \\ &= 2 \mathbf{G}^{+} \left(\mathbf{L} \otimes \mathbf{I}\right) \mathbf{F}^{\top} \mathrm{d} \mathrm{vech}\left( \mathbf{L} \right). \end{align} $$