I am not a complete beginner, but enough of a beginner that I would like some visual clarification to the mechanics that I already know. And validation that I am understanding this material.
Lets start with a simple 2d example… a line of m = -1:
\begin{align*} x + y = 0 \end{align*}
In matrix form, as a rank 1, I can write it this way:
a = \begin{bmatrix}1 & 1 \\0 & 0\end{bmatrix}
Solving this for nullspace, I use y as a free variable, and set it to 1 or 0. (when set to zero, the nullspace is only the zero vector – no problem)
when y is set to 1, x is -1.
c * \begin{pmatrix}-1\\1\end{pmatrix}
Well, the vector {-1,1} is "in the line" of the nullspace, which is the original line. And when multiplied by the matrix, gives zero…
This implies that the, and each and every row is the orthogonal vector to the nullspace?
I have never heard it explained that way.
Even expanding this to 3d: x + y + z = 0
a = \begin{bmatrix}1 & 1 & 1 \\0 & 0 & 0 \\0 & 0 & 0\end{bmatrix}
I now have two free variables, y and z.
c1*\begin{pmatrix}-1\\1\\0\end{pmatrix} c2*\begin{pmatrix}-1\\0\\1\end{pmatrix}
Again, these are two vectors, which when I plot them, they represent the span of the plane, which is the nullspace of the matrix.
So, again, these vectors, multiplied into the matrix give zero, which implies that each row is actually the orthogonal vector to the nullspace?
These span vectors are not orthogonal themselves (inner product not zero), but I know the mechanics to find orthogonal span vectors from these.
And this could be applied to a matrix where the rows do not immediately appear to be independent (to a beginner such as myself)?
d = \begin{bmatrix}3 & 4 & 5 \\9 & 12 & 15 \\12 & 16 & 19\end{bmatrix}
Because each row would have to produce a zero when multiplied by each of the nullspace vectors. So each one is perpendicular to the subspace.
In summary, much of the material I've looked at makes it appear to be a challenge to find something orthogonal to the nullspace… when in fact it is simply the rows. And they can be "factory original" or they can be "row reduced".
Am I interpreting this correctly? Or are there more things to consider?
Any comments would be appreciated.
-Scotty
Best Answer
Your observations point toward an important theorem which you will no doubt encounter soon in your studies. In one of its forms it says that the null space of a matrix is the orthogonal complement of its row space. In particular, every row of the matrix is orthogonal to every vector in its null space.
I’m not sure where you got the impression that computing orthogonal complements is particularly difficult. As you say, if you already have a matrix in hand, then its rows span the orthogonal complement of its null space, so you’re sort of done. That description of the space isn’t usually particularly useful or satisfactory, though. For practical purposes, you’re going to want a basis for this space instead of the “factory original.”
What if you’re not starting with a matrix, though? Well, if you’re working in $\mathbb R^n$ and are trying to find the orthogonal complement of some subspace for which you have a set of spanning vectors, you can set up a system of linear equations, collect the coefficients and proceed as before. In principle, you can do this for any finite-dimensional space, although generating the linear equations might take a bit more work. The definition of orthogonality depends on the inner product that you’re using—it’s not always a simple dot product of tuples of numbers. For example, your vector space could be polynomials of degree at most 5 with the inner product $\langle f(t),g(t)\rangle = \int_{-1}^1 f(t)g(t)\,dt$. Computing the orthogonal complement of the span of $t^2-1$ and $t^5+2t^3$ can still be reduced to solving a system of linear equations, but setting those equations up is a bit more involved. If the vector space is not finite-dimensional, it can be even more difficult. Think about how you might go about finding the orthogonal complement of the set of functions of the form $\alpha\sin t+\beta\sin3t$ within the space of infinitely-differentiable functions defined on the interval $[0,2\pi)$ (yes, that’s a vector space, too).