[Math] Kernel and Image of an orthogonal projection

linear algebraprojection-matrices

This is a question which I understand quantitatively but I am having a difficulty visualizing geometrically (once it comes to the kernel). I understand the explanation in the book but I still seem to struggle visualizing, thus if you have a helpful method, I would appreciate it!

Consider the orthogonal projection T(x)=proj of x onto V onto a subspace V in Rn. We are asked to find the image and kernel of this subspace.

I understand that the image is subspace V as it is composed of all the vectors (linearly independent) which span and make up the plane V. Now, the kernel is said to be the line perpendicular to V, or the normal vector to V. I know that the kernel is a subspace of V which sends all of its vectors to the null space, but how does a normal vector send all its vectors to the null space.

In summary, I am having a difficulty seeing as to how the kernel is the normal to the plane.

Best Answer

Let's restrict our attention to subspaces $V$ of $\mathbb{R}^3$ rather than $\mathbb{R}^n$. Once this case is understood, you can try to generalize it. It is important to think slowly from the definitions. Geometric intuition will come afterwards (and be correct). I will not recall the definition of orthogonal projection onto a subspace for you, you can look that up in your notes/textbook.

You appear to be confusing several concepts. Let me try to clarify them for you.

Fix a subspace $V \subseteq \mathbb{R}^3$ (this could be the origin, a line through the origin, a plane containing the origin, or the entire space $\mathbb{R}^3$). Let $T_V\colon \mathbb{R}^3 \rightarrow \mathbb{R}^3$ be the linear transformation defined by orthogonal projection onto the subspace $V$.

Any linear transformation has a kernel and an image. They are defined for $T_V$ as follows:

$$\text{image}(T_V) = \left\{ y \in \mathbb{R}^3 \colon \exists x \in \mathbb{R}^3 \text{ such that } T_V(x) = y \right\} $$

$$\text{kernel}(T_V) = \left\{ x \in \mathbb{R}^3 \colon T_V(x) = 0 \right\}$$

(you may note that both the image and the kernel of $T_V$ are subspaces of $\mathbb{R}^3$).

From the first definition, we can explain that $$\text{image}(T_V) = V.$$ The proof uses two key facts: the definition of the image of a linear transformation, and the definition of the map $T_V$.

Proof that $\text{image}(T_V) = V$: In order to do this, we show that $\text{image}(T_V) \subseteq V$ and $V \subseteq \text{image}(T_V)$:

For any vector $x \in \mathbb{R}^3$, the orthogonal projection of $x$ onto $V$ is an element of $V$. Thus $\text{image}(T_V) \subseteq V$.

On the other hand, if $x$ is an element of $V$, then $T_V(x) = x$ (the orthogonal projection of a vector in $V$ onto $V$ is itself), so $V\subseteq \text{image}(T_V)$. This completes the proof. $\square$

Thus:

  • If $V$ is a line in $\mathbb{R}^3$, then $\text{image}(T_V)$ is the same line.
  • If $V$ is a plane in $\mathbb{R}^3$, then $\text{image}(T_V)$ is the same plane.
  • If $V$ is the entire space $\mathbb{R}^3$, then $\text{image}(T_V)$ is the entire space $\mathbb{R}^3$.

Now we would like to describe the second space, $\text{kernel}(T_V)$. In order to do this, it is useful to recall that the orthogonal complement of a subspace $V$ is a new subspace defined in the following way: $$V^{\perp} = \left\{ y\in \mathbb{R}^3 : \forall x\in V, \langle x,y\rangle = 0 \right\}.$$ In plain English, $V^{\perp}$ is the set of all vectors that are orthogonal to every vector in $V$.

You should think about why the following statements are true (note that tehy only make sense if $V$ is a subspace of $\mathbb{R}^3$):

  • If $V$ is a line, then $V^{\perp}$ is a plane.
  • If $V$ is a plane, then $V^{\perp}$ is a line.
  • If $V$ is the origin, then $V^{\perp}$ is the entire space $\mathbb{R}^3$.
  • If $V$ is the entire space $\mathbb{R}^3$, then $V^{\perp}$ is the origin.

You should also try to draw pictures of some examples.

The following statement contains the intuition you are after. I will leave the proof of this to you.

$$V^{\perp} = \text{kernel}(T_V).$$