The wiki article on eigenvectors offers the following geometrical interpretation:
Each application of the matrix to an arbitrary vector yields a result which will have rotated towards the eigenvector with the largest eigenvalue.
Qn 1: If there is any other geometrical interpretation particularly in the context of a covariance matrix?
The wiki also discusses the difference between left and right eigenvectors.
Qn 2: Do the above geometrical interpretations hold irrespective of whether they are left or right eigenvectors?
Best Answer
Here is a partial answer in the case where M is a real symmetric matrix. This is to ensure, by the real spectral theorem, that M has real eigenvectors with real eigenvalues, so there is a chance for a genuine geometric interpretation which stays in $R^n$.
M acts on the unit sphere in $R^n$ in the following way: it sends the unit sphere $v^T v = 1$ to $v^T (M^T M) v = 1$ . This modified shape is not generally a sphere, but is generally an ellipsoid. The axes of this ellipsoid are the eigenvectors of M, and the sizes of each axis are given by the squares of the corresponding eigenvalues.