I am taking a graduate level course in probability and we started off with some results in functional analysis.
One thing that I feel I do not understand properly is the definition of an orthogonal projection in the context of Hilbert spaces.
And I was not able to find (sufficiently) exhaustive discussion of the definition (for example not here).
To this end, let a Hilbert space, $H$, possibly infinite dimensional and not necessarily separable. Let a closed convex subset $S \subseteq H$.
1) One the one hand orthogonal projection $P_S :H \to S$ is defined as a mapping of every element $x \in H$ to the unique best approximation of $x$ in $s_0 \in S$. That is $s_0 \in S$ such that $$\|s_0 -x\| = \inf\{s \in S| \|x- s\|\}\,. $$ (Existence of such $P_s$ eventually implies that $H = S\oplus S^{c}$)
2) On the other hand, at various places, I have seen the definition of orthogonal projection to be an operator $P:H \to H$ such that: $P^2 =P$ and $P^* = P$.
$\text{ }$
Here are two things I am trying to figure out:
A. Are those two definitions identical in an (infinite) inseparable case?
To go from 1) to 2) seems to be quite straightforward. However I could not figure out if definition 2) implies 1) in an (infinite) inseparable case.
B. The other thing, if the set $S$ is not closed under addition the operator $P$ might not be linear, so is orthogonal projection required to be linear?
I would appreciate any help.
Best Answer
Here's what you can say in general, without topology, completeness or other structure:
If $X$ is a real or complex inner product space and $M$ is a linear subspace of $X$, then define $\mathcal{D}_M$ to be the set of $x\in X$ for which there exists $m\in M$ such that $(x-m)\perp M$, and define $P_M : \mathcal{D}_M\subseteq X\rightarrow X$ so that $P_Mx=m$. It is easy to check that $P_M$ is defined for $m\in M$ and $P_M m = m$ because $(m-m)\perp M$. So $M\subseteq \mathcal{D}_M$ always holds.
Suppose $X$ is an inner product space, that $M\subset X$ is a linear subspace of $X$, and suppose that $x \in X$. For any $n =1,2,3,\cdots$, there exists $m_n \in M$ such that $$ \|x-m_n\| < \inf_{m'\in M}\|x-m'\| + \frac{1}{n}. $$ It turns out that $\{ m_n \}$ is a Cauchy sequence. Therefore, if $M$ is a complete subspace of $X$ then, then $\lim_n m_n = m$ exists, and you can show by continuity that $\|x-m\|=\inf_{m'\in M}\|x-m'\|$. Assuming $M$ is complete also gives $m\in M$ because $M$ is closed. Hence, $\mathcal{D}_M=X$ in this case, leading to an orthogonal projection $P_M : X\rightarrow X$.
A notable example where $\mathcal{D}_M= X$ is where $M$ is finite-dimensional. This follows because $M$ is complete, but you can also demonstrate this directly by choosing an orthonormal basis $\{e_n\}_{n=1}^{N}$ of $M$ and verifying that $$ \left(x - \sum_{n=1}^{N}\langle x,e_n\rangle x_n\right)\perp M. $$ So the orthogonal projection onto $M$ is $P_M = \sum_{n=1}^{N}\langle x,e_n\rangle e_n$.
If $M$ is a closed subspace of a Hilbert space $H$, then $M$ is complete and, hence, $\mathcal{D}_M=H$. In this case, $P_M$ is defined on all of $H$, is linear, and satisfies $P_M=P_M^2=P_M^{\star}$.
EXAMPLE: Let $X=L^2[0,1]$. Let $M_r = \{\chi_{[0,r]}f : f \in L^2[0,1]\}$. For each $f \in X$, it is easy to check that $$ (f - \chi_{[0,r]}f) \perp M_r. $$ Therefore $P_{M_r}f = \chi_{[0,r]}f$ and, without checking, $P_{M_r}$ must be linear and satisfy $$ P_{M_r}^{2} = P_{M_r} = P_{M_r}^{\star},\\ \|P_{M_r}f\| \le \|f\|,\;\;\; f\in L^2[0,1]. $$
Proof: For $P$ as stated, let $M=P(X)$ be the range of $P$. Then $M$ is a subspace, and, for every $x\in X$, one has $$ (x-Px)\perp M $$ because $\langle x-Px,Py\rangle = \langle Px-P^2x,y\rangle= \langle Px-Px,y\rangle=0$ for all $y\in X$. Therefore $P$ is the orthgonal projection onto $M$. $\;\;\;\blacksquare$.