Let $A$ be a matrix with linearly-independent columns. Then $A(A^TA)^{-1}A^T$ is the orthogonal projection matrix onto the range/image of $A$. This formula is legal because the linear independence of $A$'s columns ensures that $A^TA$ is nonsingular.
Now let $A$ be a linear operator in nonseparable Hilbert space $\cal H$. We do not have an orthonormal basis for $\cal H$ and we cannot necessarily find a matrix representation of $A$ with linearly-independent columns. How do we find the orthogonal projection onto the image of $A$?
My context is S.J. Bernau's The Square Root of a Positive Self-Adjoint Operator, just before lemma 16 on page 29. Bernau is proving the spectral theorem for unbounded self-adjoint operators. He shows how to construct the square root of any positive self-adjoint operator, and since $A^2$ is positive when $A$ is self-adjoint he defines an absolute value $|A|=\sqrt{A^2}$ from which he constructs the "positive" and "negative" parts of $A$ as $A^+ = {1\over2}(A+|A|)$ and $A^- = {1\over2}(A-|A|)$. Then we can decompose $A = A^+ – A^-$. If we repeat this reasoning with $A-\lambda I$ for arbitrary lambda, we get that $A – \lambda I = (A-\lambda I)^+ – (A-\lambda I)^-$, and if we keep only the positive part, we get a function that is zero whenever (in the abstract spectral sense) $A\geq \lambda I$. So the range of $(A-\lambda I)^+$ contains just those vectors that $A$ acts on with eigenvalue less than $\lambda$ (also in the abstract spectral sense), so if we can find a projection onto the range of $(A-\lambda I)^+$ we will have the projection matrix $E_\lambda$ that shows up in most statements of the spectral theorem.
All of that reasoning makes sense. The trouble I have is that the paper just defines $E_\lambda$ as the projection onto the range of $(A-\lambda I)^+$ without constructing this projection. And we can't use the spectral theorem to do it because that is what we are trying to prove!
So my precise question is: Let $B = (A-\lambda I)^+$ from the above problem. We know that $B$ is a nonnegative self-adjoint operator (possibly unbounded) on some Hilbert space. How do I define an operator that is a projection onto the image of $B$ if I am not allowed to use the spectral theorem and I cannot just assume that my operator is a finite-dimensional matrix with linearly independent columns? We are free to use any special assumptions that apply to $B$, even if they don't apply to arbitrary operators.
Update: Riesz and Nagy make the same proof in section 108 of Functional Analysis. They just say "Let $E_\lambda$ be the projection upon ${\frak L}_
\lambda$" as though this is something you can assume exists. So I'm adding the "functional-analysis" tag on the grounds that this question could be re-asked in the context of trying to understand the proof in a classic functional analysis book.
Another couple of updates, since this was solved in the comments:
- Bernau is actually referring to the null space of $(A-\lambda I)^+$, not its range. It is hard to tell because $\frak N$ and $\frak R$ look so similar. But my question is asking for the wrong projection operator!
- Riesz and Nagy have a proof in section 34 of Functional Analysis that any vector can be decomposed into the sum of an element of a subspace and a vector orthogonal to every element of the subspace. (This proof explicitly does not require the separability of the Hilbert space.) Then in section 105 they define the projection operator directly in terms of that decomposition. So this proof justifies the existence of orthogonal projections onto arbitrary subspaces, even for nonseparable spaces. So we at least have a proof that such an orthogonal projection exists.
- There is no tidy formula (like $P = A(A^TA)^{-1}A^T$) for the projection operator; I'm going to have to be happy with a formula-less proof that the operator is well-defined.
Best Answer
First of all, on page 29 of Bernau, $E(\lambda)$ is being defined as projection onto the nullspace of $(A-\lambda I)^+$, not the image. It's unfortunate that their Fraktur R and N look almost identical, but the R's have more of a crossbar and a curly tail on the lower left, so I'm pretty sure that is in fact an N. (It's also clear from context once you see what they say about it that it must be N.)
Note that $(A-I\lambda)^+$ is a closed operator (self-adjoint, even) and the nullspace of a closed operator is closed (easy exercise). And it's a general, elementary fact about Hilbert spaces that for any closed subspace $E$, there is a unique bounded operator giving orthogonal projection onto $E$. Separability is not needed. It looks like you found the relevant section in Riesz and Nagy; you can also see Orthogonal projection on the Hilbert space . here on this site.
(Just so everyone can see what I mean about the Fraktur letters, here's what they look like in the paper:
Can you tell them apart? Keep this in mind when you choose typefaces!)