Find the minimum value of $f(x)=x^TAx+b^Tx+c$

matricesmaxima-minimamultivariable-calculusquadratic programmingquadratic-forms

A scalar valued function is defined as $f(x)=x^TAx+b^Tx+c$ , where $A$ is a symmetric positive definite matrix with dimension $n\times n$ ; $b$ and $x$ are vectors of dimension $n\times 1$. Show that the minimum value of $f(x)$ will occur when $x$ equals to $-\frac{A^{-1}b}{2}$.

I found the answer of the same here. But I an unable to get the partial derivatives. Please any one explain the solution.


Related:

Best Answer

Here is a solution with no calculus at all. It's the same process of completing the square that we learn in the first algebra course in high school with some linear algebra thrown in.

Let $q=\frac12 A^{-1}b$. Note that $$f(x) = (x+q)^\top A (x+q) + (c-q^\top Aq).$$ Notice that we use symmetry of $A$ to get $q^\top Ax= \frac12 b^\top (A^{-1}Ax) = \frac12 b^\top x$ and similarly for $x^\top Aq$. Since $A$ is positive definite, $y^\top Ay\ge 0$, with equality holding if and only if $y=0$. Thus, $f$ attains its minimum when $x+q=0$, i.e., when $x=-q=-\frac12 A^{-1}b$.