I am learning linear algebra through professor Giblert Strang's lectures on MIT OCW.
The professor says that the row space and the null space of a matrix are orthogonal subspaces. This I can follow, since any vector in the nullspace takes any linear combination of the rows to zero.
He then says that the row space and null space are more than just orthogonal subspaces, they are orthogonal complements, Because The "nullspace contains all the vectors that are perpendicular to the row space", and vice versa.
Consider:
$$A= \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}=0 $$
A is rank 2, Dimension of nullspace of A=1
Null space:
$$X=c \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$$
Graphically, the row space of A is the x-y plane, and the null space is the z-axis.
It's easy to see that the nullspace does not contain all the vectors that are perpendicular to a vector in the row space. If we look at $[1 ~0~ 0]$, it has the entire y-z plane perpendicular to it, not just the z-axis.
All I can see is two orthogonal subspaces. I do not understand what additional property has earned these subspaces the term orthogonal complements.
Refer:
Lecture Video (from 31:16 to 33:00)
Lecture Notes Page 2 paragraph 1
Best Answer
Let $$U= \operatorname{span}\{(1,0,0)\} \\ V = \operatorname{span}\{(0,1,1)\} \\ W=\operatorname{span}\{(0,1,0),(0,0,1)\}$$ Then clearly every vector in $U$ is orthogonal to every vector in $V$ and to every vector in $W$. Hence $U$ and $V$ are orthogonal subspaces and $U$ and $W$ are orthogonal subspaces.
But $U$ and $V$ together don't make up all of the vectors in $\Bbb R^3$ (the ambient space). For instance $(0,0,1)$ is not in either one. However $U$ and $W$ together do in fact make up all of the vectors in $\Bbb R^3$. In the lingo of linear algebra, we say that the direct sum of $U$ and $W$ is $\Bbb R^3$ -- or symbolically $U\oplus W = \Bbb R^3$.$^\dagger$ It's this extra property that makes $U$ and $W$ not just orthogonal subspaces -- but each other's orthogonal complement.
$\dagger$: I'm being a little loose with my language here. Technically the condition for $U\oplus W=\Bbb R^3$ (which is the required extra condition) is that every vector in $\Bbb R^3$ can be written in a unique way as a sum of an element of $U$ and of $W$. I.e. for every $v\in \Bbb R^3$, there exists a unique $u\in U$ and a unique $w\in W$ such that $u+w=v$. Nevertheless, the statements I give in the paragraph above are how we intuitively think of orthogonal complements.