[Math] How to eigenvectors and eigenvalues possibly be so useful


I'm having a hard time trying to pin down why calculating the eigenvectors and eigenvalues of a matrix almost always gives something useful. There are numerous answers here on SX that discuss how to calculate them and their utility in the context of specific applications, but I haven't been able to find any answers that specifically discuss how or why these relatively simple operations produce useful answers in so many different disciplines.

For example,

  • In analysis (data science), the eigenvalues and eigenvectors of the covariance matrix give you an orthogonal basis with each eigenvector accounting for as much variance as possible and the eigenvalues tell you how much variance that eigenvector "captures" (i.e., principal components analysis, PCA). More importantly, it's a dimension reduction technique that enables you to treat high-dimensional data in an approximate way so that you might still get a useful answer out of it.
  • In quantum mechanics, the eigenvectors of an operator (represented as a matrix) give you the eigenstates of the system and the eigenvalues tell you how likely you are to measure it in that state.
  • In mechanics, the eigenvectors of the moment of inertia matrix tell you the principal axes around which an object will rotate (the eigenvalues probably give you something useful, as well, but I can't remember).
  • In social network analysis, eigenvector centrality is a way of calculating the "influence" of a node.

I have the feeling that there must be some reason why eigenvectors and eigenvalues are so broadly applicable. What is it that this operation does that almost always results in something useful coming out?

Edit, clarification: Mechanically, the procedure is always the same. Formulate a problem as having a set of inputs and generate a matrix of values that are a function of the pairwise combinations of the inputs. Then, calculating the eigenvectors and eigenvalues of this matrix will (typically) give you something useful.

What, specifically, does calculating eigenvectors and eigenvalues of this matrix do that gives you something useful?

Best Answer

I dont know if this is the intuition you are looking for, but basically eigenvectors and diagonalization uncouple complex problems into a number of simpler problems.

In physics often stress or movement in one direction, the $x$ direction will cause a stress or movement in the $y$ and $z$ direction. An eigenvector is a direction where stress or movement in the eigendirection remain in the eigendirection, thus chosing an eigen basis replaces a complex $3$dimensional problem by three $1$-dimensional problems.

You talk about pairwise combinations of inputs, eigenvectors simplify this into function of just one input.

Look up weakly coupled oscillatory systems in a dynamics book. Systems wich complex oscillations are analysed into eigenvectors with periodic oscillations.

Related Question