So here is a direct passage from the book we are learning from.
Every vector $\boldsymbol x$ in the nullspace is perpendicular to every row of $A$, because $A\boldsymbol x=\boldsymbol 0$. The nullspace $N(A)$ and the row space $C(A^{\rm T})$ are orthogonal subspaces of $\,{\bf R}^n$.
To see why $\boldsymbol x$ is perpendicular to the rows, look at $A\boldsymbol x=\boldsymbol 0$. Each row multiplies $\boldsymbol x$:
$$A\boldsymbol x=\begin{bmatrix}\text{row }1\\\vdots\\\ \quad\!\text{row n}\quad \end{bmatrix}\begin{bmatrix}\\ \boldsymbol x \\ \,\,\end{bmatrix}=\begin{bmatrix}0\\\vdots\\0\end{bmatrix}\quad\matrix{\longleftarrow&\; \text{(row 1)}\cdot\boldsymbol x\text{ is zero}\\ \\ \longleftarrow&\; \text{(row $m$)}\cdot\boldsymbol x\text{ is zero} }
\tag1$$
Now the image I get in my mind is that A belongs to column space, A(transpose) belongs to row space and that x belongs to null space/left nullspace depending on we are calculating column or row space. This is the books best description of what represents what. I am trying to get an image of my head how things are related.
The image below is what I'm trying to figure out, x is in nullspace/left nullspace and the vectors a1 and a2 that belong to Matrix A is in rowspace and columnspace. Is this an accurate thinking of how the vectors are represented when x is nullspace/left nullspace if trying to visualize them?
Best Answer
There is no point in trying to visualise this since the row space of $A$ and its null space are not naturally subspaces of the same space. If $A$ corresponds to a linear map $V\to W$, then the null space of $A$ corresponds to the kernel $\ker(f)$, a subspace of$~V$. Each row of $A$ computes one coordinate of the image of $f$, so it is a linear form on $V$ (a linear function $V\to\Bbb R$). The row space is defined by the linear combinations of these rows, so it is naturally a subspace of the space $V^*$ of linear forms on$~V$. This space $V^*$ not the same space as$~V$, even though it has the same dimension; it is called its dual vector space. The row space is then a subspace $R\subseteq V^*$, and the natural statement of what the cite passage is saying is that $\ker(f)$ is the set of vectors where all linear forms of$~R$ are zero simultaneously. This is quite unsurprising, since the null $\ker(f)$ is by definition the set where the linear forms corresponding to the rows of $A$ all are zero (the intersection of their zero sets), but once this happens for some $v\in V$, any linear combination of those linear forms is also zero at$~v$.
Now imagining combinations of linear forms on $V$ might be a bit hard, so one may just imagine that $V$ has an inner product for which the basis that one was using (to define the matrix $A$ of $f$) is orthonormal. This inner product is artificial and has no clear meaning, but at least it allows to represent each linear forms as the inner product with one specific vector$~v_i$. This representing of linear forms by vectors is what goes on in the operation of transposition. Then row$~i$ of $A$ corresponds to (the inner product with)$~v_i$, and the row space corresponds to the span of $v_1,\ldots,v_m$. Now the subspace where the linear form for row$~i$ of$~A$ is zero is the set of vectors orthogonal to$~v_i$. Then $\ker(f)$ is the set of vectors where this happens for all rows at once, i.e., the subspace of vectors orthogonal to $v_1,\ldots,v_m$, which is the orthogonal complement of the space of $v_1,\ldots,v_m$. This is a way to visualise, but remember that we are just saying that all linear forms in $R$ are zero simultaneously.