It makes intuitive sense to me that the very definition of the null space – all x's that produce the zero vector when multiplied by the rows of a matrix A – would coincide with the conditions for orthogonality (dot product of two vectors = 0). But how would one show that this applies for all combinations of A? I guess i'm trying to wrap my head around a more rigorous way of understanding the notion that the row space is the orthogonal compliment to the null space of a matrix… I hope this makes sense and I am looking forward to hearing some insights!
[Math] How would one prove that the row space and null space are orthogonal compliments of each other
linear algebraorthogonality
Related Solutions
The orthogonal projection $P_V$ to a subspace $V$ is the unique linear transformation such that:
- for any $v\in V$ we have $P_V(v)=v$ and
- for any $u\in V^\perp$ we have $P_V(v)=0$.
Now, for $V$ the column space of $Q$:
- $v\in V$ means $v=Qw$ for some $w$. Then $QQ^Tv=QQ^TQw=Qw=v$
- $u\in V^\perp$ means $u\in \ker Q^T$. Then $QQ^Tu=Q0=0$.
Thus $P_V=QQ^T$.
For a more general matrix $A$ with linearly independent columns, projection onto its column space is given by $A(A^TA)^{-1}A^T$ for the same reason. This is a standard formula in "least squares".
The above reasoning only allows us to check the answer once we know it. To come up with it in the first place, we can reason as follows. Let column space of $Q$ be $V$. Then the rows of $Q^T$ are a set of relations that need to hold for a vector to be orthogonal to $V$. That is, the rows of $Q^T$ record all the things that need to vanish for a vector to be in $V^\perp$. So any linear map $P$ vanishes on $V^\perp$ precisely when it "factors through" $Q^T$, that is when it is a composition of something with $Q^T$, i.e. $P=MQ^T$.
Now, in order for $P$ to then be identity on $V$ itself, we must have $PQw=Qw$, so we must have $MQ^TQw=Qw$. But for us $Q^TQ=Id$. So $M=Q$ and so $P=QQ^T$.
It is essentially just about counting the dimensions. Say that $A$ is $m\times n$ matrix, and the first $k$ rows of $R_A$ are non-zero and the remaining $l=m-k$ rows are zero. We observe two facts about $R_A$:
$\mathrm{rank}(R_A)=k$; This holds because $R_A$ is in row echelon form, and so its non-zero rows are linearly independent. And they obviously generate the row-space.
$\mathrm{null}(R_A)=l$, where $\mathrm{null}(R_A)$ is the dimension of the left null-space of $R_A$. This is true because the null space is $l$-dimensional space consisting of exactly those vectors that are zero in the first $k$ coordinates.
Now the key fact is that multiplying by $E$ does not change the dimensions of the row space and the null-space -- this needs a bit of thought, but it essentially boils down to the fact that $v\mapsto v\cdot E$ is surjective, and injective, respectively. In particular, we have $\mathrm{null}(A)=l$.
The last $l$ rows of $E$ are all in the left null-space of $A$, because $E_i\cdot A=(R_A)_i=0$ for $i=r+1,\dots, r+l$. On the other hand, those rows are linearly independent, because $E$ is invertible. So for dimensional reasons, $(E_{r+1},\dots,E_{r+l})$ must be a basis of the null space.
Remark: I didn't use the rank-nullity theorem, because the question essentially leads to the proof of it, so I felt like it might be a circular argument that way.
Best Answer
Note that matrix multiplication can be defined via dot products. In particular, suppose that $A$ has rows $a_1$, $a_2, \dots, a_n$, then for any vector $x = (x_1,\dots,x_n)^T$, we have: $$ Ax = (a_1 \cdot x, a_2 \cdot x, \dots, a_n \cdot x) $$ Now, if $x$ is in the null-space, then $Ax = \vec 0$. So, if $x$ is in the null-space of $A$, then $x$ must be orthogonal to every row of $A$, no matter what "combination of $A$" you've chosen.