Exercise on Orthogonal Projection

functional-analysishilbert-spacesorthogonalityprojection

For an exercise, I need to investigate the orthogonal projection in more detail. I know that this has probably been discussed many times here, I am though interested in whether my explanations suffice and in particular in Tasks 3 and 4. Concretely, the exercise states :

Let $\mathcal{M}$ be a closed subspace of a Hilbert space $\mathcal{H}$ and let

$\mathcal{M}^{\bot} = \{x \in \mathcal{H} : \langle x, y \rangle_{\mathcal{H}} = 0 , \forall y \in \mathcal{M} \}$

be the corresponding orthogonal space, i.e. the space of all vectors $x \in \mathcal{H}$ which are perpendicular to all vectors $y \in \mathcal{M}$. Then, it can be shown that every vector $x \in \mathcal{H}$ has a unique decomposition $x = x_{\mathcal{M}} + x_{\bot}$ with $x_{\mathcal{M}} \in \mathcal{M}$ and $x_{\bot} \in M^{\bot}$. Based on this decomposition, one defines the mapping $P_{\mathcal{M}} : \mathcal{H} \rightarrow \mathcal{H}$ by

$P_{\mathcal{M}}(x) = x_{\mathcal{M}} , x \in \mathcal{H}$

Tasks :

  1. Is $P_{\mathcal{M}}$ linear?
  2. What is the range $R(P_{\mathcal{M}})$ and the nullspace $\mathcal{N}(P_{\mathcal{M}})$ of this mapping?
  3. Determine the norm of $P_{\mathcal{M}}$
  4. Verify that
    $P_{\mathcal{M}}(x) = \arg \min_{y \in \mathcal{M}} \left\| x – y \right\|_{\mathcal{H}}$
    , i.e. $\hat{x} = P_{\mathcal{M}} x$ is the vector that minimizes $\left\| x – y \right\|_{\mathcal{H}}$ among all $y \in \mathcal{M}$.

My thoughts so far :

  1. It is easy to see that $P_{\mathcal{M}}(\alpha x_1 + \beta x_2) = (\alpha x_1 + \beta x_2)_{\mathcal{M}} = \alpha \cdot (x_1)_{\mathcal{M}} + \beta \cdot (x_2)_{\mathcal{M}}$ via the properties of the linear subspace $\mathcal{M} \subset \mathcal{H}$.
  2. The orthogonal projection is a projection for which the range and nullspace are orthogonal subspaces. Hence, we have that the range $R(P_{\mathcal{M}}) = \mathcal{M}$ as defined above. Consequently, the nullspace must be $\mathcal{N}(P_{\mathcal{M}}) = M^{\bot}$.
  3. The norm can be derived as follows. Consider $\left\| Px \right\|^2 = |\langle Px, Px \rangle| = |\langle x, Px \rangle| \leq \left\| x \right\| \cdot \left\| Px \right\|$. Accordingly, we have that $\left\| P \right\| = \frac{\left\| Px \right\|}{\left\| x \right\|} \leq 1$
  4. By inspection, it becomes clear that $y$ is the solution to the problem, since perpendicularity implies that the scalar product in the corresponding space is zero. But how to verify that??

I would appreciate a quick look at my results as well as an intuition as how to verify task 4. Any help is appreciated 🙂

Best Answer

  1. You haven't checked that $P$ is a projection (i.e. $P^2 = P$) so you'll need to prove this if you want to use properties of projections. If I were you I would just prove the results directly $$ \text{Im } P = \{ P(x_M + x_\perp) = x_M : x_M \in M, x_\perp \in M^\perp \} = M $$ $$ \ker P = \{ x = x_M + x_\perp: x_M = 0, x_\perp \in M^\perp,x_M \in M\} = M^\perp$$

  2. $\left\| P \right\| = \frac{\left\| Px \right\|}{\left\| x \right\|}$ is not how the operator norm is defined. Also $\Vert P\Vert \leq 1$ is not sufficient to determine the norm of $P$ you must still show that $\Vert P \Vert \geq 1$ to deduce that $\Vert P \Vert = 1$. You're better off using the identity $P^2 = P$ for this part

  3. Hint : use the fact that if $v $ and $w$ are orthogonal then $$ \Vert v+w \Vert^2 = \Vert v \Vert^2 + \Vert w \Vert^2$$ (this is just Pythagoras' theorem). Use this with $x - y = (x_M - y) + x_\perp$

Related Question