[Math] Proving full column rank of a matrix

linear algebramatricesmatrix-rank

Let $x$ be a $K\times 1$ vector of random variables satisfying that $E[xx']$ is nonsingular. For some given integers $M\geq 1$ and $L\leq K$, let $z_1,\ldots,z_M$ be $L\times 1$ column vectors such that each $z_m$ contains distinct elements from $x$. I would like to understand that
$$
\Sigma\equiv E\begin{pmatrix}xz_1'\\ \vdots \\ xz_M'\end{pmatrix}
$$
has full column rank. Can you please help?

I can vaguely imagine that I probably need to write $z_i=D_ix$ for some $D_i$ that picks out the rows of $x$. But I can't proceed.

Edit: This problem appears as Review Question 5.1 in Hayashi (2000).

Best Answer

$\Sigma$ has $K\cdot M$ rows and $L$ columns. Since $L\leq K,$ the statement "$\Sigma$ has full column rank" means $rank\ \Sigma = L.$

Put $$ \Xi := E(xz_1'), $$ i.e. $\Xi$ consists of the first $K$ rows of $\Sigma.$ By construction, $\Xi$ has $L$ rows. If we can show $rank\ \Xi = L,$ then the $L$ columns of $\Xi$ are linearly independent. This implies that the $L$ columns of $\Sigma$ are also linearly independent, which means $rank\ \Sigma \geq L.$ Since we have $rank\ \Sigma \leq L$ (by looking at the dimensions of $\Sigma$), we can conclude $rank\ \Sigma = L,$ as desired. So it remains to show that $rank\ \Xi = L.$

But this is "easy to see" as follows. By the assumptions on $z_1,$ $\Xi$ is a $K$-by-$L$ submatrix of $E(xx') =: \Theta.$ More specifically, $\Xi$ is obtained from $\Theta$ by deleting some columns. Since $\Theta$ is invertible (by assumption), every subset of its columns is linearly independent. This means $$ rank\ \Xi = L, $$ as desired.

Related Question