Distance between the spans of two matrices

linear algebramatricesmatrix-normsnormed-spacesprojection-matrices

We have two matrices of the same dimension, $V_1$ and $V_2$. We take
$$
P_i=V_i(V_i^tV_i)^{-1}V_i^t.
$$
Then we take the Frobenius norm (Hilbert Schmidt norm) of $P_1-P_2$. Why does this not change even if one changed any of the $V_i$ to $V_i'$ as long as they span the same space. Moreover, I always get this distance to be between 0 and 2 when $V_i$ has two columns. Why is that? Thank you.

You may wish to see julia code demonstration here.

The Frobenius norm of a matrix is the square root of the sum of the squares of the elements.

Feedback from Kusma.
$$
VM(M^tV^tVM)^{-1}M^tV^t
$$
is same as $V(V^tV)^{-1}V^t$ for $M$ full rank. So first part is solved. Thanks.

Best Answer

$P_1$ and $P_2$ are projections ($P_i^2=P_i$), so they are determined by the space they project onto. That the squared distance is bounded by the rank (I assume that the distance will get larger if you take $5\times4$ matrices instead of $4\times2$) is connected to the fact that all eigenvalues of projections are $0$ or $1$.

Your projections also satisfy $P_i^t=P_i$, so the matrix $P_1-P_2$ is diagonalisable with respect to some orthonormal basis. The eigenvalues all satisfy $-1\le \lambda_j\le 1$, so the square of the Frobenius norm is bounded by the size of the matrix.

Related Question