I'm following along Ch. 4 entitled "Orthogonality" in Gilbert Strang's Introduction to Linear Algebra. Here are some of the initial results in that chapter
-
Definition: two subspaces of a vector space are orthogonal if every vector in the first subspace is perpendicular to every vector in the second subspace
-
Every vector $x$ in the nullspace of a matrix $A$ is perpendicular to every row of $A$.
-
Similarly, every vector $y$ in the nullspace of $A^T$ is perpendicular to every column of $A$.
-
Definition: The orthogonal complement of a subspace $V$ contains every vector that is perpendicular to $V$
By this definition, the nullspace is the orthogonal complement of the
row space. Every $x$ that is perpendicular to the rows satisfies
$Ax=0$.
My question is about this last quote.
How do we know that all vectors not in the nullspace are in the row space?
Every vector $x$ that satisfies $Ax=0$ is in the nullspace of $A$, and is perpendicular to every row of $A$. Thus, $x$ is also perpendicular to linear combinations of rows of $A$ and is thus perpendicular to every vector in the row space.
If $A$ is $m$ by $n$ with rank $r$, the nullspace has dimension $n-r$ and the row space has dimension $r$. Vectors in the nullspace and in the row space are in $\mathbb{R}^n$.
How do we know that not only are these two subspaces orthogonal, they are also orthogonal complements because there is no vector in $\mathbb{R}^n$ that is perpendicular to the vectors in the nullspace that is not in the row space, and there is no vector perpendicular to the vectors in the row space that is not in the null space?
Best Answer
The boldface question/statement is incorrect. No one is asserting that the complement of the nullspace is the rowspace. The claim is that the nullspace is the orthogonal complement of the rowspace.
As you note, anything in the nullspace is orthogonal to the rowspace. If $v$ is orthogonal to the rowspace, then it is orthogonal to each row, so we also have $Av=\mathbf{0}$. Thus, we have that the nullspace is contained in the orthogonal complement of the rowspace (by the first observation), and that the orthogonal complement of the rowspace is contained in the nullspace (by the second observation). Thus, the nullspace is equal to the orthogonal complement of the rowspace.
Alternatively, we know by the Rank-Nullity Theorem that the dimension of the rowspace plus the dimension of the nullspace is $n$. In addition, the dimension of the rowspace plus the dimension of the orthogonal complement of the rowspace also add up to $n$. Since the nullspace is contained in the orthogonal complement, and they must have the same dimension by the previous computations, we conclude they have the same dimension and thus are equal, since one is contained in the other.