Consider the following quantity
$$X^T (XX^T + \mathrm{Id})^{-1} X,$$
where $X \in \mathbb{R}^{m\times n}$ is a iid random matrix with 0 mean and finite variance.
The empiric covariance matrix ${X^T X}$ has been extensively studied and the asymptotic (when $n,m\rightarrow \infty$ and $n/m\rightarrow \gamma$) law of its eigenvalues is for instance knwon as the Marchenko-Pastur distribution after adequate renormalizations.
Is there a generalization or similar analyses for the case of the matrix introduced above, which naturally arise in the context of nested linear models as a modification of $X^T X$?
Best Answer
Since $$Z=X^\top (XX^\top + \mathrm{Id})^{-1} X=(Y + \mathrm{Id})^{-1} Y,$$ with $Y=X^\top X$, we can find the eigenvalue density of $Z$ from the eigenvalue density of $Y$, which for large matrix dimensions is given by the Marchenko-Pastur density $\rho_{\rm MP}(y)$.
In particular, if the matrix elements of $X$ have variance $1/m$, the average $\langle z\rangle$ of an eigenvalue of $Z$ is given by $$\langle z\rangle=\int_0^\infty \frac{y}{1+y}\,\rho_{\rm MP}(y)\,dy=\frac{\sqrt{\gamma^2+4}+\gamma+2}{2 \gamma},$$ in the limit $n,m\rightarrow\infty$ at constant $\gamma=n/m\in(0,1]$.