[Math] How to find max and min of this quadratic form

linear algebramatricesoptimization

So I'm a little confused about how to finish of this homework question and even unsure if my attempt is correct so far. . .

" Find an orthonormal basis of matrix A" = $$\begin{pmatrix} 2 & 1 & 1 \\ 1 & 1 & 0 \\ 1 & 0 & 1\\ \end{pmatrix}$$

so I found the eigenvalues to be

$\mathbb c=0, 1, 3 $ and the respective normalized eigenvectors to be $$\mathbb 1/\sqrt3 \begin{pmatrix} -1\\ 1 \\ 1\\ \end{pmatrix} \mathbb 1/\sqrt2 \begin{pmatrix} 0\\ -1 \\ 1\\ \end{pmatrix} \mathbb 1/\sqrt6 \begin{pmatrix} 2 \\ 1 \\ 1 \\ \end{pmatrix}$$

I think this ok, they seem to be orthogonal.

For the quadratic form $\mathbb q(x)=(Ax, x)$ find the maximal and minimal value of $\mathbb q(x)$ on the unit spher $\mathbb S = \{x| (x,x) = 1\}$

For this I'm stumped, I know it clearly must involve what I've done and I've read questions about quadratic forms which say there is diagonalizable matrix with eigenvectors on the diagonal such that max and min are the max an min eigenvalues but how I apply that to my question I'm really unsure. Is it just the clearly can't be the eigenvectors I found and obviously not just expected to do it through partial derivatives, can anyone just how I may proceed ? How does the definition of unit sphere come into it?

are the eigenvalues and vectors of A I found correct AND involved in the 2nd part?

Best Answer

What you have done so far appears correct to me.

Here is the idea without resorting to diagonalization: the $v_j$ form a basis , and so you can write every $x = \sum c_j v_j$. By linearity,

$$ Ax = A(\sum c_j v_j) = \sum c_j Av_j = \sum c_j \lambda_j v_j. $$

Then

$$ \left< x,Ax\right> = \left< \sum c_i v_i ,A\sum c_j v_j \right> = \left< \sum c_i v_i ,\sum c_j \lambda_j v_j \right> = \sum_{i,j}c_i c_j \lambda_j \left<v_i,v_j\right> = \sum_j c_j^2 \lambda_j $$

since $\left<v_i,v_j\right> = 1$ if $i = j$ and $0$ otherwise. In particular, how can you choose the $c_j$ so that $\sum (c_j)^2 \lambda_j$ is at its maximum or minimum?

If you want to know how this relates to diagonalization (something you should eventually learn about), $A$ being symmetric ensures it has orthogonal eigenvectors, and that you can diagonalize $A$

$$ A = QDQ^T $$

where the columns of $Q$ are your eigenvectors, and $D$ is a diagonal matrix with the eigenvalues on the diagonal. Now, take any $x$, and write $x = \sum c_j v_j$ where $v_j$ are the eigenvectors you found. Note that $Q^Tv_j = e_j$, the standard basis vectors. Also $De_j = \lambda_j e_j$ and $Qe_j = v_j$. Thus,

$$ x^TAx = x^TQDQ^Tx = x^TQD\left(\sum c_j e_j\right) \\ = x^TQ\left(\sum \lambda_j c_j e_j\right) \\ = x^T\left(\sum \lambda_j c_j v_j\right) \\ = \sum (c_j)^2 \lambda_j. $$

You must have $\sum (c_j)^2 = 1$ so that $x$ is in the unit sphere (this is only required because a quadratic form does not have a global max or min: $\left<\alpha x , A(\alpha x) \right> = \alpha^2 \left<x,Ax\right>$, and so we just fix a particular scale ($1$ in this case)).