Show there exists a global minimum

calculuslinear algebraoptimization

Let $A\in R^{n\times n}$ be a symmetric positive definite matrix and $b\in R^n$. Define $f:R^n\to R$ by $f(x)=\frac{1}{2}x^TAx-b^Tx$. Show that $f$ is strictly convex and has exactly one global minimum.

I've managed to prove that $f$ is strictly convex through some messy calculation. But I only need the fact that $A$ is positive definite to show convexity. So I suppose to prove there exists a global minimum I need to use $A$ being symmetric. How do I do that? And since $f$ strictly convex, if it has a minimum, it will be unique, right?

Best Answer

If you are allowed to use an excursion into linear algebra, you can prove your claim quite easily as follows using:

  • $A$ is symmetric, so there is an orthogonal matrix $U$ with $$U^TAU = \operatorname{diag}(\lambda_1,\ldots , \lambda_n)$$
  • $A$ is positive definite, so $\lambda_i>0\,(i=1,\ldots , n)$
  • Using the fact $U^TU = UU^T = I_{n\times n}$, and setting $y= U^Tx$ and $c=U^Tb$, rewrite $f(x)$ as follows $$\frac{1}{2}x^TAx-b^Tx = \frac{1}{2}y^T\operatorname{diag}(\lambda_1,\ldots , \lambda_n)y-c^Ty = \frac{1}{2}\sum_{i=1}^n\left(\lambda_iy_i^2 -2c_iy_i \right)$$ $$= \frac{1}{2}\sum_{i=1}^n\left(\left(\sqrt{\lambda_i}y_i -c_i\right)^2-c_i^2\right) =:g(y)$$

$g(y)$ is easily checked to be strictly convex and has a unique global minimum at $y_i =\frac{c_i}{\sqrt{\lambda_i}}\, (i=1,\ldots , n)$.