Linear Algebra: solving minimization problems using orthogonal projections

linear algebraorthogonality

I'm learning about the application of orthogonal projections in solving minimalization problems, and need help clarifying the example below.

Page 113 below lists out some definitions and theorems, and the bottom half of page 114 is the example.

enter image description here
enter image description here
enter image description here

My questions are as follows:

  1. What does the expression $C = U \oplus U^\perp$ mean? From the definitions of $C$ and $U$, I understand that $U$ is a subspace of $C$, but intuitively I don't get what $U^\perp$ is, and how its direct sum with $U$ is the whole $C$. To use the example in the text, if $v = sin(x)$ is in $C$, and $u$ is a polynomial with real coefficients and degree at most 5, what is $u^\perp$ and how are the three things related?

  2. Why the inner product 6.39? Can we use any inner product?

  3. How are the two ideas marked in red related? In particular, in my understanding we are considering the whole interval $[-\pi,\pi]$, and I don't see how this information is encoded in the simple expression $||v – u||$.

Best Answer

  1. By definition$$U^\perp=\{v\in V\,|\,(\forall u\in U):\langle u,v\rangle=0\}.$$It's therefore the space of all elements of $V$ which are orthogonal to each element of $U$. And $U\cap U^\perp=\{0\}$, from which it follows that you can form a direct sum with them.
  2. Because it is with this inner product that you have $\|f\|=\sqrt{\langle f,f\rangle}$, where$$\|f\|=\sqrt{\int_{-\pi}^\pi\bigl\lvert f(x)\bigr\rvert^2\,\mathrm dx}$$
  3. Because, in this context,$$\|u-v\|=\sqrt{\int_{-\pi}^\pi\bigl\lvert u(x)-v(x)\bigr\rvert^2\,\mathrm dx}.$$
Related Question