It's known that a sum $\mathbf S_n$ of iid random vectors $\mathbf {X_1,X_2, X_3,…X_n} $ which are $\mathbb{R^d} $ normaly distributed with covariance matrix $\Sigma$ and mean $\mathbf 0 $, will converge in distribution by the following relation :
\begin{equation}
(\frac{\mathbf S_n}{\sqrt{n}})\rightarrow N_d(0,\Sigma).
\end{equation}
My question regards the dependence between the d elements of $\mathbf X$. By considering the covariance matrix $\Sigma$, aren't we looking at the linear dependence only? What if instead we consider more complex dependence structures, are they going to be preserved as the covariance is preserved above?
My research into this matter involved consulting a few profs of the multidimensional CLT which I don't fully grasp. My questions are : *1)*Why the covariance structure is preserved? *2)*Why is it the only thing needed? (what is the relation to Cramer-Wold theorem?) *3)*Why, if I assume more then linear relation between the elements of $\mathbf X$ that extra dependence structure is destroyed?
Any help would be greatly appreciated. Thank you.
Edit :The main question is this : If you start with a given copula structure between the elements of $\mathbf X$ you than end up invariantly with a dependence structure given by the implied normal copula of $N(0,\Sigma)$. Why isn't the dependence structure preserved?
Best Answer
First off, it is not necessary that the variables $(X_k)_{k\ge1}$ are normal. The requirement (in the classical CLT, at least) is that the variables $(X_k)_{k\ge1}$ are independent and identically distributed with a distribution with mean zero and finite variance $\Sigma$. As you write, the conclusion
$$ S_n / \sqrt{n} \to N_d(0,\Sigma) $$
then holds. Given this, I'll try to answer your questions in the sequence (3), (2), (1).
(3): As I understand your question, you find that a linear relationship between the coordinates in the distribution of the $X_k$'s is necessary for the CLT to hold? As in the CLT cited above, this is not necessary. The distribution can be anything, as long as the mean is zero and the variance is finite. Thus, the CLT holds "for any dependence structure".
(2): Well, the CLT is often proven by showing pointwise convergence of characteristic functions and using a Taylor expansion around zero. In the Taylor expansion, only the first and second moments figure, since the first and second order derivatives of the characteristic function is directly related to the first and second order moments of the probability distribution. I guess that's sort of an informal way of explaining why only first and second moments are necesary.
(1): It's sort of a loose question, I guess my best answer would be: If the covariance of the limit distribution shouldn't be the common covariance $\Sigma$ of $S_n/\sqrt{n}$, what else should it be...? Preservation of moments is the most natural outcome for a regular limit situation like this.