Simple question.
We know from the fundamental theorem of linear algebra that the nullspace of a matrix is the orthogonal complement of its row space.
I can write this as:
Let $M$ be a matrix. The following two conditions are equivalent:
(i) $u$ is orthogonal to the null space of $M$.
(ii) $u$ is in the row space of $M$.
That (ii) implies (i) is trivial, pretty much by definition.
Question: But why is it also true (or even obvious) that (i) implies (ii)?
Best Answer
First, I'll prove/outline/mention a few preliminary results.
Lemma 1: If $V$ is a finite-dimensional real-vector space and $W$ is a subspace of $V,$ then for all $v\in V,$ there exist unique $w\in W,w'\in W^\perp$ such that $v=w+w'$.
Proof: It is readily seen that existence implies uniqueness, since if $w_1,w_2\in W$ and $w_1',w_2'\in W^\perp$ such that $w_1+w_1'=w_2+w_2',$ then $w_1-w_2=w_2'-w_1',$ but $w_1-w_2\in W$ and $w_2'-w_1'\in W^\perp,$ so since $W\cap W^\perp$ is the zero subspace (the zero vector is the only self-orthogonal vector), then $w_1-w_2=w_2'-w_1'=0,$ so $w_1=w_2$ and $w_1'=w_2'$. To prove existence, we can use the Gram-Schmidt process, starting with a basis for $W,$ to make an orthonormal basis for $W,$ which we then extend to an orthonormal basis for $V$ (possible in finite dimensions), and the added vectors will be an orthonormal basis for $W^\perp.$ $\Box$
Lemma 2: If $V$ is a real-vector space and $W$ is a subspace of $V,$ then $(W^\perp)^\perp\supseteq W$. [Readily seen by definition.] $\Box$
Lemma 3: If $V$ is a finite-dimensional real-vector space and $W$ is a subspace of $V,$ then $(W^\perp)^\perp=W.$
Proof: Take any $v\in(W^\perp)^\perp,$ and write $v=w+w'$ as in Lemma 1. Since $v\in(W^\perp)^\perp$ and $w'\in W^\perp,$ then $$0=v\cdot w'=(w+w')\cdot w'=w\cdot w'+w'\cdot w',$$ and so $0=w'\cdot w'$ since $w\in W$ and $w'\in W^\perp.$ Only the zero vector is self-orthogonal, so $w'=0,$ so $v=w\in W$, giving us the reverse inclusion of Lemma 2. $\Box$
Lemma 4: Given a finite dimensional vector space $V$ and subspaces $W$ and $X$ of $V$, we have that $W^\perp=X^\perp$ if and only if $W=X$.
Proof: One implication is trivial. For the other, suppose $W^\perp=X^\perp.$ Take $x\in X$. By Lemma 1, there exist unique $w,w'$ such that $x=w+w',$ $w\in W$, and $w'\in W^\perp=X^\perp.$ Then $$0=x\cdot w'=(w+w')\cdot w'=w\cdot w'+w'\cdot w'=w'\cdot w',$$ so $w'=0,$ whence $x=w\in W$, and so $X\subseteq W$. By symmetrical arguments, we likewise have $W\subseteq X$, so $W=X$. $\Box$
Now, because of Lemmas 3 and 4, to show that (i) implies (ii) is equivalent to showing that every vector in the null space of $M$ is orthogonal to every element of the row space of $M$. Indeed, Say that $M$ is an $m\times n$ matrix, and writing $M=[r_1^T\:\cdots\:r_m^T]^T$ where the $r_j$ are the rows of $M,$ we note that for any $n$-dimensional vector $x$ we have $$Mx=\left[\begin{array}{c}r_1\\\vdots\\r_m\end{array}\right]x=\left[\begin{array}{c}r_1x\\\vdots\\r_mx\end{array}\right]=\left[\begin{array}{c}r_1^T\cdot x\\\vdots\\r_m^T\cdot x\end{array}\right].$$ In particular, if $x$ is in the null space of $M$, then $r_j^T\cdot x=0$ for $1\le j\le m,$ meaning that $x$ is orthogonal to each row of $M$, so since the row space of $M$ is spanned by these rows then $x$ is in the orthogonal complement to the row space of $M,$ as desired.