I need an upper bound for a quadratic form of the type:
$$x^T A x \leq \lambda_{\max}(A) \| x \|^2,$$
where $A$ is a real symmetric and non-singular matrix.
I recall that this can be done by defining $\lambda_{\max}(A)$ to be the maximum eigenvalue of $A$. Is this correct? If so, where can I find this result?
Best Answer
It's equivalent to analyze $$\max_{x\in \mathbb{R}^{n}\setminus \mathbf{0} } \frac{x^T A x}{x^T x}$$ Form linear algebra there exist $\Omega \in O(n)$ such that $$\Omega^T A \Omega = \text{diag} \{\lambda_1,\lambda_2,\cdots,\lambda_{n}\}$$ where $\lambda_1,\lambda_2,\cdots,\lambda_n$ are real numbers . And hence $$ \max_{x\in \mathbb{R}^{n}\setminus \mathbf{0} } \frac{x^T A x}{x^T x} = \displaystyle \max_{\sum_{j=1}^{n} y_j^2 = 1} \sum_{j=1}^{n} \lambda_i y_i^2 $$ It's easy to see $$\max_{\sum_{j=1}^{n} y_j^2 = 1} \sum_{j=1}^{n} \lambda_i y_i^2 \le \max \lambda_i$$ and the equality holds when $x=\Omega^Ty=\Omega^T(y_1,y_2,\cdots,y_n)^T$ is the eigenvector of $\max \lambda_i$.
Thus $$ \max_{x\in \mathbb{R}^{n}\setminus \mathbf{0} } \frac{x^T A x}{x^T x}= \max \lambda_i $$