Does Rayleigh quotient iteration always find the largest eigenvalue in magnitude? If not, what are the applications

Question

I'm self-studying numerical linear algebra. It seems like, Rayleigh quotient iteration doesn't always guarantee to find the biggest eigenvalue (in terms of magnitude). If we start from an random guess of vector $v$, Rayleigh quotient iteration will bring me to the eigenvalue whose corresponding eigenvector would be closest to $v$.

Is my understanding correct? In that case, I wonder what are the main applications of Rayleigh quotient iteration? I'm under the impression that most of the time we really want to find the largest eigenvalues. The largest eigenvalues give us the best approximation of a matrix. Finding a random eigenvalue, which might happen to be a tiny one, seems rather pointless?

Answer 1

The initial guess of a vector is not as important as the initial guess of an eigenvalue, actually. If the initial guess $\mu_0$ in Rayleigh quotient iteration is sufficiently close to a true eigenvalue $\lambda$, and if the initial vector $b_0$ is not nearly orthogonal to the $\lambda$-eigenspace, then the iterative method will actually find the eigenvalue $\lambda$ and one of its eigenvectors. So the eigenvalue we find is not entirely random.

Still, not being able to find the eigenvalues in order is a disadvantage, but this algorithm makes up for it by having much faster (cubic) convergence!

There are two solutions to the disadvantage:

• You could use a few steps of power iteration to get pretty close to the largest eigenvalue/eigenvector, then continue with Rayleigh quotient iteration to take advantage of the fast convergence. This is reasonable if all you want is the largest eigenvalue.
• If your ultimate goal is to find all the eigenvalues, then you can pick them off one by one with Rayleigh quotient iteration. Simply start with a guess $b_0$ orthogonal to all previously found eigenvectors $x_1, \dots, x_k$. (If necessary, periodically replace $b_i$ by $b_i - \sum_{j=1}^k \frac{\langle b_i, x_j\rangle}{\langle x_j, x_j\rangle} x_j$ to preserve that orthogonality.)