I am currently working on showing that $\Vert Px \Vert_{2} \leq \Vert x \Vert_{2}$, where $x \in$ an inner product space $X$, and $P$ is the orthogonal projection operator. Also, I am supposing that $X$ has a subspace $U$ with orthonormal basis $\{u_{1},u_{2},\cdots,u_{n}\}$, and thus that $P:X \to U$ is defined by $Px = \sum_{i=1}^{n}\langle x, u_{i} \rangle u_{i}$.
To show that $\Vert Px \Vert_{2} \leq \Vert x \Vert_{2}$ for all $x \in X$, I am proceeding as follows:
$\Vert P x \Vert_{2}^{2} = \vert \langle Px, Px \rangle \vert = \vert \langle x, Px \rangle \vert \leq \Vert x \Vert_{2}\,\, \Vert Px \Vert_{2}$
My only concern is that I'm justified in saying that $Px = x$ inside the inner product operator for the first components. I've read that that's a property of orthogonal operators, but it also looks an awful lot like the statement that $Px = x$ for $x \in U$, which is something that needs to be proven, AND is true ONLY for $x \in U$, not for $x \in X \backslash U$, so it's making me uncomfortable.
Can someone please let me know if my discomfort is justified, and if so, how I show that $Px=x$ for $x$ in ALL of $X$ as well?
Thank you!
Best Answer
Given a subspace $U\subseteq X$, the space may be decomposed as $X=U\oplus U^\perp$. For example, in three dimensions, given a line $\ell$, there is a perpendicular plane $\pi$, and every vector may be uniquely written as a vector from $\ell$ plus a vector from $\pi$. Two key properties of the inner product are that it is linear in both arguments (or, if you're working with a complex vector space, it is conjugate-linear in the left or right argument if you are a physicist or mathematician respectively) and that it evaluates to zero whenever its arguments are orthogonal, by definition of orthogonal.
(By the way, it's a nice exercise to prove $X=U\oplus U^\perp$.)
In particular, given any $x,y\in X$, we may write $x=u+u'$ and $y=v+v'$ where $u,v\in U$ and $u',v'\in U^\perp$, and then $\langle x,y\rangle=\langle u+u',v+v'\rangle=\langle u,v\rangle+\langle u,v'\rangle+\langle u',v\rangle+\langle u',v'\rangle$ which simplifies to just $\langle u,v\rangle+\langle u',v'\rangle$ since the other two inner products evaluate to zero. More specifically we can say that $\|u+u'\|^2=\|u\|^2+\|u'\|^2$ when $u\in U,u'\in U^\perp$.
Take for instance $X=\mathbb{R}^2$ and $U$ the first coordinate axis. Then the orthogonal projection just shift's every vector's tail vertically until it's on the first coordinate axis. If our vector is $(3,4)$ then the projection $P$ pushes it down to $(3,0)$, which is the $U$-component of $(3,4)$, and the $U^\perp$ component of $(3,4)$ must then clearly be $(0,4)$. Draw all this out - it's a right triangle by virtue of the fact the projection is orthogonal. This is the picture behind $\|Px\|\le \|x\|$.
Indeed the identity $\|u+u'\|^2=\|u\|^2+\|u'\|^2$ is none other than the Pythagorean theorem!
To prove it formally, you've started out on the right path. It's easier to work with the square of magnitude since it comes from a linear gadget (the inner product), and an inequality of nonnegative reals is equivalent to what you get by squaring. So work with $\|x\|^2$. First note that we can decompose $x=Px+Qx$ (the $U\oplus U^\perp$ decomposition of $x$, where $Q:X\to U^\perp$ is the orthogonal projection onto $U^\perp$), and now since $Px\perp Qx$ we can say
$$\| x\|^2=\|Px+Qx\|^2=\|Px\|^2+\|Qx\|^2\ge \|Px\|^2 \implies \|x\|\ge \|Px\|. $$
Note that it's a necessary condition that $P$ is an orthogonal projection. Otherwise, if we pick an $x$ that is orthogonal to $\ker P$ but not in $U$, the vectors $x$ and $Px$ are once again the edges of a right triangle but with $Px$ the hypotenuse and $x$ a side, giving $\|Px\|>\|x\|$ in that case.