Introduction to Statistical Learning Eq. 4.32

linear algebramachine learningmatrix

Can someone please explain how the third line becomes the fourth line?
enter image description here

Best Answer

It is an issue of expanding and tidying up. You have for example

  • $(x-\mu_k)^T\Sigma^{-1}(x-\mu_k) = x^T\Sigma^{-1}x - \mu_k^T\Sigma^{-1}x-x^T\Sigma^{-1}\mu_k+\mu_k^T\Sigma^{-1}\mu_k$ and
  • $\mu_k^T\Sigma^{-1}x=x^T\Sigma^{-1}\mu_k$ and $\mu_k^T\Sigma^{-1}\mu_K=\mu_K^T\Sigma^{-1}\mu_k$ and
  • $(\mu_k+\mu_K)^T\Sigma^{-1}(\mu_k-\mu_K) = \mu_k^T\Sigma^{-1}\mu_k -\mu_K^T\Sigma^{-1}\mu_K$

so

$\log\left(\frac{\pi_k}{\pi_K}\right) -\frac12(x-\mu_k)^T\Sigma^{-1}(x-\mu_k) +\frac12(x-\mu_K)^T\Sigma^{-1}(x-\mu_K)$
$= \log\left(\frac{\pi_k}{\pi_K}\right) -\frac12 x^T\Sigma^{-1}x+ x^T\Sigma^{-1}\mu_k- \frac12\mu_k^T\Sigma^{-1}\mu_k +\frac12 x^T\Sigma^{-1}x- x^T\Sigma^{-1}\mu_K+ \frac12\mu_K^T\Sigma^{-1}\mu_K$
$= \log\left(\frac{\pi_k}{\pi_K}\right) - \frac12(\mu_k^T\Sigma^{-1}\mu_k - \mu_K^T\Sigma^{-1}\mu_K) + x^T\Sigma^{-1}\mu_k- x^T\Sigma^{-1}\mu_K$ $= \log\left(\frac{\pi_k}{\pi_K}\right) - \frac12\mu_k^T\Sigma^{-1}\mu_k -\mu_K^T\Sigma^{-1}\mu_K + x^T\Sigma^{-1}(\mu_k- \mu_K)$

Related Question