I have a clear understanding of how the column space relates to a transformation of the basis vectors, but I really do not see the connection between the column space and the row space of the matrix.
If we have $3 \times 3$ transformation matrix $A$ defined by
$$
A = \begin{pmatrix}
a_{11} & a_{12} & a_{13} \\
a_{21} & a_{22} & a_{23} \\
a_{31} & a_{32} & a_{33}
\end{pmatrix}
$$
We have the basis for the column space given by
$$
\left( \vec{c_1}\begin{pmatrix}
a_{11} \\
a_{21} \\
a_{31}
\end{pmatrix}, \vec{c_2}\begin{pmatrix}
a_{12} \\
a_{22} \\
a_{32}
\end{pmatrix}, \vec{c_3}\begin{pmatrix}
a_{13} \\
a_{23} \\
a_{33}
\end{pmatrix} \right)
$$
Then the operation on a $3 \times 1$ column vector $\vec{v}$ is defined as
$$
A \cdot \vec{v} = \begin{pmatrix}
a_{11} & a_{12} & a_{13} \\
a_{21} & a_{22} & a_{23} \\
a_{31} & a_{32} & a_{33}
\end{pmatrix} \cdot \begin{pmatrix}
x \\
y \\
z
\end{pmatrix} = \begin{pmatrix}
xa_{11} + ya_{12} + za_{13} \\
xa_{21} + ya_{22} + za_{23} \\
xa_{31} + ya_{32} + za_{33}
\end{pmatrix}
$$
Which is equivalent to the linear combination of the basis vectors for the column space $\left( \vec{c_1}, \vec{c_2}, \vec{c_3} \right)$ such as
$$A \cdot \vec{v} = x\vec{c_1} + y\vec{c_2} + z\vec{c_3}$$
Or the dot product with the basis vectors for the row space as shown in the matrix-vector product. We have the basis for the row space given by
$$
\left( \vec{r_1}\begin{pmatrix}
a_{11} \\
a_{12} \\
a_{13}
\end{pmatrix}, \vec{r_2}\begin{pmatrix}
a_{21} \\
a_{22} \\
a_{23}
\end{pmatrix}, \vec{r_3}\begin{pmatrix}
a_{31} \\
a_{32} \\
a_{33}
\end{pmatrix} \right)
$$
And the resulting matrix-vector product given by
$$
A \cdot \vec{v} = \begin{pmatrix}
\vec{r_1} \cdot \vec{v} \\
\vec{r_2} \cdot \vec{v} \\
\vec{r_3} \cdot \vec{v}
\end{pmatrix}
$$
My question concerns the relationship between the linear span of the column space and the linear span of the row space.
Is there any geometric relationship between the column space and the row space apart from this algebraic manipulation that emphasizes the fact the linear combination of the basis vectors for the column space $\left( \vec{c_1}, \vec{c_2}, \vec{c_3} \right)$ is equal to the projection onto the basis vectors for the row space $\left( \vec{r_1}, \vec{r_2}, \vec{r_3} \right)$?
I am looking for a more geometric interpretation rather than the standard algebraic relationship.
Best Answer
One nice interpretation of this phenomenon comes out of studying the bilinear form $f:\Bbb R^3 \times \Bbb R^3 \to \Bbb R$ defined by $f(x,y) = x^TAy$.
Note that for any fixed $x \in \Bbb R^3$, the function $f(x,\cdot): \Bbb R^3 \to \Bbb R$ defined by $f(x,\cdot)(y) = f(x,y)$ gives us a linear map (an element of the dual space to $\Bbb R^3$, if you prefer). We can identify the column space of $A$ with all maps of the form $f(x,\cdot)$; each column of $A$ corresponds to plugging in a standard basis vector for $x$. Similarly, we can identify the row space of $A$ with all maps of the form $f(\cdot,y)$; each row of $A$ corresponds to plugging in a standard basis vector for $y$.
In a sense, your observation amounts to the statement that $$ f(x, \cdot)(y) = f(\cdot,y)(x). $$