I have been trying to find a good source (or clarifications) to help me understand this point. I am very new to random matrix theory so any pointers will be appreciated. Here is what I think I have understood so far
Consider N, P-dimensional random vectors drawn from a multivariate normal distribution and stacked into a matrix $X$ ($X$ will have dimension $N \times P$).The covariance matrix $S = \frac{1}{N}X^{*}X$ will have Wishart distribution with $\Sigma$, the $P \times P$ covariance matrix of the underlying multivariate normal referred to as the support. The question is how do the eigenvectors of this sample covariance matrix look?
Under certain conditions (which I am trying to understand as part of this question) ALL eigenvectors of the random sample covariance matrix are expected to be uniformly distributed on a unit sphere (see http://projecteuclid.org/download/pdfview_1/euclid.aop/1181334252 on pg 6/41 and in this thesis https://rucore.libraries.rutgers.edu/rutgers-lib/40727/ on pg 26)
(1) Could someone provide a reference to this result and
(2) is this true for any covariance matrix? Most studies are interested in only the diagonal support covariance matrix case and it makes me wonder if the result is applicable for more general supports.
(3) Even in the diagonal case, they take $\Sigma = \sigma^{2}I$. Will the result hold in the diagonal support case if the entries on the diagonal are not the same i.e. $\Sigma = diag[\sigma_{1},\sigma_{2}…]$?
Best Answer
Assume $\Sigma = I_p$. The eigenvectors of $X^TX$ are the right singular vectors of $X$, e.g., $v_1,...,v_r$ with $r=\min(n,p)$ if the SVD of $X$ is $X=\sum_{i=1}^r s_i u_i v_i^T$.
You wish to prove that $v_i$ is uniformly distributed on the sphere. The key is that for any deterministic or random rotation matrix $Q\in O(p)$, $X$ and $XQ$ ar equal in distribution (this follows immediately for instance by observing that the rows of $X$ and $XQ$ are both iid $N(0,I_p)$).
This implies that for every $i=1,...,r$, the $i$-th singular vector $v_i$ is equal in distribution to the $i$-th singular vector of $XQ$ which is no other than $Q^Tv_i$ since the SVD of $XQ$ is $$XQ = \sum_i s_i u_i v_i^T Q = \sum_i s_i u_i (Q^Tv_i)^T.$$ Hence $v_i$ is equal in distribution to $Q^Tv_i$. Note that we made no assumption on $Q$ except that it belongs to $O(p)$ and is independent of $X$. In particular we would like to choose a random $Q\in O(p)$ such $Q^Tu$ is uniformly distributed on the sphere for every deterministic $u$ with $\|u\|=1$; if such $Q$ exists then in our SVD problem, $Q^Tv_i$ (which is equal in distribution to $v_i$) is thus uniformly distributed on the sphere.
Of course, the follow up question is how to construct such random matrix $Q$. This distribution is called the Haar measure of the orthogonal group $O(p)$.
If $\Sigma\ne I_p$ this is not true. For instance if $\Sigma= a a^T$ is rank one, the only right singular vector of $X$ is nonrandom and equal to $a/\|a\|$.