In the first week of Linear Algebra I learned that we could represent a system of equations, such as $$x_1 + 4x_2 + 7x_3 = 10 \\
2x_1 + 5x_2 + 8x_3 = 11 \\
3x_1 + 6x_2 + 9x_3 = 12$$
as an augmented matrix of the form $$\begin{bmatrix}
1 & 4 & 7 & 10 \\
2 & 5 & 8 & 11 \\
3 & 6 & 9 & 12 \end{bmatrix}$$
We then learned that if $\vec{v_1} = \begin{bmatrix} 1 \\
2 \\
3 \end{bmatrix}, \vec{v_2} = \begin{bmatrix} 4 \\
5 \\
6 \end{bmatrix}, \vec{v_3} = \begin{bmatrix} 7 \\
8 \\
9 \end{bmatrix}$, and $\vec{b} = \begin{bmatrix} 10 \\
11 \\
12 \end{bmatrix}$, then the vector equation $$x_1 \vec{v_1} + x_2 \vec{v_2} + x_3 \vec{v_3} = \vec{b}$$ can be solved using the same augmented matrix since it forms the equivalent system of equations. That is, the coefficients that tell you how to add up combinations of $\vec{v_1}, \vec{v_2}$, and $\vec{v_3}$ to get $\vec{b}$ are the exact same values that solve the intersection of the three planes in the system of equations.
My question is thus: Beyond the algebra just working out, is there a solid, intuitive, geometric reason why when we take the x-values of $\vec{v_1}, \vec{v_2}$, and $\vec{v_3}$ and make a plane with them, take the y-values of $\vec{v_1}, \vec{v_2}$ and make a plane with them, and $\vec{v_3}$, take the z-values of $\vec{v_1}, \vec{v_2}$, and $\vec{v_3}$ and make a plane with them, that the intersection of those special planes gives us our solution? (And to be more precise, that those values act as the normal vectors for three planes)
Again, I understand that algebraically the system of equations and the vector equation are the same so it has to work out…but I can't seem to be able to make a visual connection if I were to draw the $x_1$–$x_2$–$x_3$ axis and graph the three planes, and then draw the x-y-z axis and the vectors $\vec{v_1}, \vec{v_2}, \vec{v_3}$ and $\vec{b}$, how would I be able to see that they're even related at all!
Best Answer
The great teacher of linear algebra, Gilbert Strang, refers to these two different ways of visualizing a system of linear equations as the "row picture" and the "column picture". Professor Strang encourages his students to learn to think in terms of the "column picture", because it is more tractable when your vector space has more than 3 dimensions.
I think you already grasp the "row picture": think of $x_1$, $x_2$, and $x_3$ as coordinates in a 3D space. Each of your 3 equations (such as $x_1 + 4x_2 + 7x_3 = 10$) designates a subset of points which satisfy the equation; namely, a plane. The point(s) which satisfy all three equations are at the intersection of the three planes, and they are your solution(s).
Next is the "column picture". Now $x_1$, $x_2$, and $x_3$ are scaling factors which you apply to 3 particular vectors to make the sum of the scaled vectors reach a particular point in space.
In the "row picture", each point in the imagined space represented a possible solution to the system of equations. But in the "column picture", each point in the imagined space is a set of RHS values for the system. With the "row picture", perturbing the RHS of the equations would mean moving the planes a little bit in their tangent directions, so they intersect at a slightly different place. But with the "column picture", perturbing the RHS would mean moving the "target point" a bit, so you need slightly different scaling factors to make your vector sum reach that point.
Now we come to your question: how can you intuitively grasp that the "row picture" and "column picture" are equivalent?
I don't know how your brain works, so I can only tell you what is helpful to me in that regard. YMMV.
One thing that might be helpful is to imagine how "searching for a solution" would look in both row and column pictures. Imagine you start with random values of $x_1$, $x_2$, and $x_3$, and have to adjust them to get closer and closer to a solution. Say you start by adjusting $x_1$ first. In the row picture, you are moving your random starting point along one axis only. In the column picture, you are scaling just one of your 3 summand vectors (i.e. moving the sum of the scaled vectors in the direction of the 1st vector only).
Do you see the parallels between those two ways of visualizing the situation? In the row picture, if you move your starting point along one axis only, you can likely hit all 3 of the planes, but not at the same time. As you move the point towards one plane, you will likely be moving away from another one. In the column picture, if you scale only one summand vector, you can likely match the X, Y, and Z coordinates of the target point, but not all at the same time. If you scale $v_1$ to match the X coordinate of the target point, you will likely miss the Y coordinate, and so on.
After you play this game a bit, you would recognize that it works better to adjust two values at the same time. In the row picture, once you have found one of the three planes, you could adjust two coordinate values at once (maintaining a specific ratio) so you stay on that plane while searching for the other planes. In the column picture, once you have matched one coordinate of the target point, you can scale two vectors at once (maintaining a specific ratio) so you continue matching that coordinate while trying to match another one.
Is this helpful? Perhaps you could also think about what perturbing one particular RHS value would look like in both row picture and column picture, and try to see that the effect on the solution is the same in both cases. Then think about what perturbing one particular LHS coefficient would look like in both row and column pictures, and try to see that the effect on the solution is the same in both cases.
You could also think about cases where there is no solution to a system of linear equations. What does that situation look like in both row and column pictures? (It is completely analogous.) Then think about cases where there are an infinite number of solutions. What does that situation look like in both row and column pictures? (Again, it's completely analogous.)
In conclusion: Don't give up! Once you really grasp both the row picture and column picture of linear systems, you will have made a significant step towards understanding linear algebra!