I'm trying to analytically find the gradient the log a multivariate Gaussian density, which is given by
$$
f(x_{1}, \dots, x_{p}) = f(\vec{x}) = \frac{1}{(2\pi)^{p/2}|\Sigma|^{1/2}} \exp \Bigg(-\frac{1}{2}(\vec{x} – \vec{\mu})^{T}\Sigma^{-1}(\vec{x} – \vec{\mu})\Bigg) \sim \mathcal{N}(\vec{\mu}, \Sigma)
$$
where $\vec{x}, \vec{\mu} \in \mathbb{R}^{p}$, and $\Sigma \in \mathbb{R}^{p \times p}$.
I want to find the analytical expression for $\nabla f$, where $\nabla$ is the gradient operator defined as
$$
\nabla f(\vec{x}) = \Bigg[ \frac{\partial f(\vec{x})}{\partial x_{1}}, \dots, \frac{\partial f(\vec{x})}{\partial x_{p}} \Bigg]
$$
My vector calculus is very rusty and not sure how to take partial derivatives with respect to $x_{1}, \dots, x_{p}$.
Any help is much appreciated!
Best Answer
Hint:
The relevant part is $$\exp\left(\frac12 \sum_{i,j=1}^p (\Sigma^{-1})_{ij}(x_i-\mu_i)(x_j-\mu_j)\right)$$
so its $k$-th partial derivative is
$$\exp\left(\frac12 \sum_{i,j=1}^p (\Sigma^{-1})_{ij}(x_i-\mu_i)(x_j-\mu_j)\right) \cdot \frac\partial{\partial x_k}\left(\frac12 \sum_{i,j=1}^p (\Sigma^{-1})_{ij}(x_i-\mu_i)(x_j-\mu_j)\right) \\= \exp\left(\frac12 \sum_{i,j=1}^p (\Sigma^{-1})_{ij}(x_i-\mu_i)(x_j-\mu_j)\right)\cdot \frac12 \left(\sum_{i,j=1}^p (\Sigma^{-1})_{ij}\delta_{ik}(x_j-\mu_j) + \sum_{i,j=1}^p (\Sigma^{-1})_{ij}\delta_{jk}(x_i-\mu_i)\right)$$ Since $\Sigma^{-1}$ is symmetric, the two sums are equal so the final result is $$\exp\left(\frac12 \sum_{i,j=1}^p (\Sigma^{-1})_{ij}(x_i-\mu_i)(x_j-\mu_j)\right) \cdot\left(\sum_{i=1}^p (\Sigma^{-1})_{ik}(x_i-\mu_i)\right)$$