Suppose you have a matrix :
$$A = \begin{pmatrix} 13 & -4 & 2 \\ -4 & 13 & -2 \\ 2 & -2 & 10 \end{pmatrix}.$$
I want to find the maximum and minimum values of the ratio
$$\frac{x'Ax}{x'x}$$ where $x =(x_1,x_2,x_3)$ is nonzero. Is there a way you can figure this out manually?
Best Answer
Since it's a symmetric matrix whose entries are real, it can be diagonalized by an orthogonal matrix $G$. That is the spectral theorem. And remember: an orthogonal matrix is a square matrix $G$ with real entries whose inverse is its transpose, so you have $G'G = I$. So you get $$ A= G'\begin{bmatrix} \lambda_1 & 0 & 0 \\ 0 & \lambda_2 & 0 \\ 0 & 0 & \lambda_3 \end{bmatrix}G = G'\Lambda G. $$ By permuting rows of $G$, you can do this in such a way that $\lambda_1>\lambda_2>\lambda_3$.
Consequently $$ \frac{x'Ax}{x'x} = \frac{x'(G'\Lambda Gx}{x'(G'G)x} = \frac{(x'G')\Lambda(Gx)}{(x'G')(Gx)} = \frac{(Gx)'\Lambda (Gx)}{(Gx)'(Gx)} = \frac{y'\Lambda y}{y'y} =\frac{\lambda_1 y_1^2 + \lambda_2 y_2^2 + \lambda_3 y_3^2}{y_1^2+y_2^2+y_3^2}. $$ If $(y_1,y_2,y_3)=(c,0,0)\ne(0,0,0)$, then the value of the fraction is $\lambda_1$, and similar comments apply to the other two $\lambda$s. What remains is to show that if more than two coordinates of $(y_1,y_2,y_3)$ are nonzero, then the value of the fraction is a weighted average of the $\lambda$s, so the value is somewhere between what it would be in the opposite extreme cases.
The values of $\lambda_i$, $i=1,2,3$ are eigenvalues, and are therefore the zeros of the characteristic polynomial.