Are there known generalisations of the central limit theorem for several random variables when the covariance matrix is degenerate?
The usual proof of CLT based on characteristic functions (see e.g. Wikipedia) yields a degenerate multivariate normal distribution. I'm after any kind of result that resolves that degeneracy. Before taking the limit one evidently has a well behaved probability distribution for the sum of $N$ instances of each of the random variables, and one may anticipate that the width of the distribution of the sum in the degenerate directions is less than $\sqrt{N}$. But can more be said in general?
Best Answer
That the covariance matrix (say $\Sigma$) is degenerate just means that the corresponding random vector $X$ (say in $\mathbb{R}^k$) lies almost surely in a proper linear subspace $V$ of $\mathbb{R}^k$, say of dimension $r<k$. Replace the random vector $X$ by the $r$-tuple (say $Y$) of the coordinates of $X$ in an arbitrary basis $B$ of $V$. Then the covariance matrix of $Y$ will be nonsingular, and so, the multivariate CLT will be applicable -- to iid copies $Y^{(1)},\dots,Y^{(n)}$ of the vector $Y$ in $\mathbb{R}^r$.
The subspace $V$ can be described as the orthogonal complement to $\mathbb{R}^k$ of the null space of $\Sigma$. For the basis $B$ you can take any set of orthonormal eigenvectors (say $e_1,\dots,e_r$) of $\Sigma$ corresponding to the nonzero eigenvalues (say $\lambda_1,\dots,\lambda_r$) of $\Sigma$. Then $X=\sum_{j=1}^r Y_je_j$ and $Y=(Y_1,\dots,Y_r)$, with $Y_j=\langle e_j,X\rangle$, where $\langle\cdot,\cdot\rangle$ denotes the inner product in $\mathbb{R}^k$. The (nonsingular) covariance matrix of $Y$ will then be the diagonal matrix with $\lambda_1,\dots,\lambda_r$ on its diagonal.