Let $D = \{u \in \mathbb{R}^n | \|u\|=1\}$ denote the unit sphere.
Let $A \in \mathbb{R}^{n \times n}$ be a symmetric matrix, and denote its smallest eigenvalue by $\lambda_\min$. Why do we have
$$\lambda_\min =\min_{u \in D} u^TAu?$$
I don't quite get what the minimum imply in this statement, does it mean that the smallest unit vector $u$ or smallest value of $u^TAu$ given $u$ is a unit vector?
Best Answer
Let $\;\lambda_1\le\ldots\le\lambda_n\;$ be the eigenvalues of $\;A\;$ , and let $\;\{v_1,...,v_n\}\;$ be an orthonormal basis of corresponding eigenvectors of $\;A\;$. Let us denote by $\;(,)\;$ the usual inner product in $\;\Bbb R^n\;$ .
Thus, if we take $\;x=\sum\limits_{k=1}^na_kv_k\in\Bbb R^n\;$ , then we get (all this stuff is around the spectral theorem. Check this important theorem) for the quadratic form $\;q(x):=x^tAx\;$ the following:
$$q(x)=x^tAx=(Ax,x)=\sum_{k=1}^n\lambda_ka_k^2\ge\lambda_1\sum_{k=1}^na_k^2=\lambda_1\left\|x\right\|^2$$
The above is true for any $\;x\in\Bbb R^n\;$ and then also for any unit vector there. Finally, observe that
$$\lambda_1=\lambda_1\left\|v_1\right\|=\lambda_1(v_1,v_1)=v_1^tAv_1$$
...and of course $\;v_1\;$ is a unit vector. Now fill up details and finish the argument (there's hardly anything left)