I'm working through Gilbert Strang's Introduction to Linear Algebra book and am really confused by a paragraph from chapter 4.1 titled 'Orthogonality of the Four Subspaces'. The paragraph is as follows:
Every vector goes to the column space! Multiplying by A cannot do anything else. More than that: Every vector $b$ in the column space comes from one and only one vector $x_r$ in the row space. Proof: If $Ax_r = Ax'_r$, the difference $x_r – x'_r$ is in the nullspace. It is also in the row space, where $x_r$ and $x'_r$ came from. This difference must be the zero vector, because the nullspace and row space are perpendicular. Therefore $x_r = x'_r$.
Further on in the book an exercise is given, where we have to demonstrate this using the following figure: Two pairs of orthogonal subspaces, with the following matrix: $A = \begin{bmatrix}1 & 2\\3 & 6\end{bmatrix}$. The column space of the matrix is: $(1, 3)$, and its row space is: $(1, 2)$. If I multiply A with the randomly chosen $x$ vector: $(1, 1)$, I arrive at $b = (3, 9)$. However, this $b$ seems unable to be recreated using a multiple of the row space vector: $(1, 2)$. I'm really confused by this. I also feel like I'm missing the meaning of the proof and am not familiar with the $'$ symbol in $Ax'_r$. Does it mean the transpose?
Any help would be greatly appreciated!
Best Answer
This theorem is strange, because its not always true... It only holds when the matrix $\mathbf{A}$ has full rank. So probably context is missing here.
Anyway, to your question: The row space that is spanned by your example matrix is NOT $$\text{span}\left(\begin{bmatrix}1\\2\end{bmatrix}\right),$$ it is $$\text{span}\left(\begin{bmatrix}1\\2\end{bmatrix}, \begin{bmatrix}3\\6\end{bmatrix}\right).$$
You have two linear independent rows (this is important for the theorem to work!), so you can span $\mathbb{R^2}$.
But the underlying meaning here is: $$\mathbf{A}\begin{bmatrix}x_1\\x_2\end{bmatrix}=\mathbf{A}_1x_1+\mathbf{A}_2x_2$$ That means that regardless of what you put in as $x$, you will get a linear combination of the columns of $\mathbf{A}$, so you are in the column space of $\mathbf{A}$.
The theorem now says that if you columns are linear independent, for each element in the column space there is exactly one $\begin{bmatrix}x_1\\x_2\end{bmatrix}$ that will lead to this vector by computing $\mathbf{A}\begin{bmatrix}x_1\\x_2\end{bmatrix}$.
If that is not true, then you have a null space of $\mathbf{A}$, but the null space will always be orthogonal to the column space of $\mathbf{A}$.
I hope that cleared some things up. If not, please ask!