linear algebraorthogonalitypseudoinverse
I am reading Gilbert Strang's "Linear Algebra and its Applications 2nd Edition".
He wrote "All solutions of $A \overline{x} = p$ share this same component $\overline{x_r}$ in the row space, and differ only in the nullspace component $w$".
I cannot understand why this fact is true.
Please tell me the proof.
The core of studying matrices is to study linear transformations between vector spaces. These can be realized as matrix multiplication on the left (or right) of column (or row) vectors.
If we are in this setup: $x\mapsto Ax$ for a column vector $x$ and appropriate matrix $A$, then the image of the linear transformation will be spanned by the columns of $A$.
The kernel of the transformation (nullspace) is the set of all $x$ such that $Ax=0$ is important for understanding the solutions to some matrix equations. You probably have already learned that if $x_0$ is a solution to $Ax=b$, then every other solution is given by $x_0+k$ where $k$ is in the nullspace.
This all has analogous explanation on the other side. If we are in this setup: $x\mapsto xA$ for a row vector $x$, then the image of the linear transformation is now spanned by the rows of $A$.
Talking about the nullspace of $A^T$ is just a fancy way of dressing up the "left nullspace" of $A$, since $xA=0$ iff $A^T x^T=0$. The nullspace is now the set of all $x$ such that $xA=0$, and you can draw the same conclusions about solutions to $xA=b$.
In short, these four spaces (really just two spaces, with a left and a right version of the pair) carry all the information about the image and kernel of the linear transformation that $A$ is affecting, whether you are using it on the right or on the left.
I think the reason the diagram is drawn the way it is in the text is to underscore that the row space and the nullspace are orthogonal complements, and similarly that the column space and the left nullspace are orthogonal complements. The order of terms (e.g. y is to the right of $A^T$) is not something he is trying to illustrate would be my guess: Focus on the depiction of orthogonality between the subspaces instead. Also notice that a major part of the illustration (at least in my version) includes arrows connecting the spaces that depict what he calls the 'true action of the matrix A.' I think your "flipside" picture could be used as a complementary illustration, but it wouldn't be correct with the remaining parts of the original diagram in which the arrows map the orthogonal dissection of A into rowspace and nullspace on the left to orthogonal dissection of column space and left nullspace on the right. I realize that you wanted to omit orthogonality from the discussion but I really think that is the issue (Strang's diagram is a synthesis of sections 2.4-2.5 and is meant to encompass these ideas) so it is impossible to interpret without considering the other aspects. Here's a photo of the diagram from my 2nd edition of Strang so you can see the arrows (I'm curious if this diagram has changed in other editions of Strang's book?):
Best Answer
Suppose that $\bar x$ satisfies $A \bar x = p$. Let $w = \bar x - \bar x_r$, and notice that $A w = A \bar x - A \bar x_r = p - p = 0$. This shows that $\bar x$ can be written as $$ \bar x = \bar x_r + w, $$ where $w \in N(A)$.