I am studying functional analysis and orthogonal projections creep up every once in a while in the textbook I am using. After looking over some of the concepts, I am beginning to realize that I may not have as good of an understanding of orthogonal projections (or projections in general for that matter) as I thought from when I studied linear algebra. I have tried looking up some resources, but what I usually find is the projection of a vector onto another, as opposed to the more general notion of projecting along some subspace.
My current understanding is as follows. Suppose $U$ is some subspace of a vector space $V$ over a field $\mathbb{F}$, and suppose that $\{u_n\}$ forms an orthogonal basis for $U$. This implies that for any vector $v \in V$, we may write
$$v = a_1u_1 + \ldots + a_nu_n + z,$$
where $a_i \in \mathbb{F}$ and $z \in U^\perp$. Then the orthogonal projection of $v$ along $U$ is a map such that $v \mapsto \hat{v}$, where
$$\hat{v} = a_1u_1 + \ldots + a_nu_n.$$
Is this the correct picture or have I misunderstood something?
If this is indeed correct, then I have a few follow-up questions. Namely, how is guaranteed that $V$ can be decomposed into $U$ and $U^\perp$. Also, is a nonorthogonal projection one where the basis vectors of $U$ are not orthogonal (in my example, that would mean $\{u_n\}$ are not necessarily orthogonal). Lastly, how does this picture imply that any projection matrix is idempotent?
Best Answer
You need an inner product to talk about orthogonality, and you need the space to be complete and $U$ to be closed to be able to write $V=U+U^\perp$. That is, you need a Hilbert space.
The existence of the decomposition is shown to exist by proving that $\{u_n\}$ can be extended to an orthogonal (usually one does orthonormal, but it is not essential). This gives the decomposition $U+U^\perp$, which allows you to define the projection. It is an idempotent because when you write $x=x_U+x_{U^\perp}$ this decomposition is unique. Then $$P^2x=P (Px)=Px_U=x_U=Px,$$ so $P^2=P$.
A non-orthogonal idempotent will be one coming from a decomposition $V=U+U'$, where $U'\cap U=\{0\}$ but $U'$ is not orthogonal to $U$.