Let $\mathscr{B} := \mathbb{R}^n$ equipped with the euclidian norm,
let $M \in \operatorname{Mat}_{n,m}(\mathbb{R})$ a Matrix with
$\operatorname{rank} M = n \le m$ and $\mathscr{A} := \{ Mx \mid x \in
\mathbb{R}^m\}$, thus $\mathscr{A}$ is the linear subspace spanned by
the linear span of the column vectors of $M$; let $f \in \mathscr{B}$.Show that there is a unique best approximation on $f$ in
$\mathscr{A}$, in other words there is an element $a^* \in
\mathscr{A}$ that holds$$ \Vert a^* -f \Vert \le \Vert a -f \Vert \ \ \ \forall a \in
\mathscr{A}$$and it holds that $$ a^* = M(M^\top M)^{-1}M^\top f$$
Hint: $a^* – f$ is orthogonal to the linear subspace $\mathscr{A}$
So i started getting into some approximation theory and was trying to solve some exercises in order to get an idea about certain topics. This one is an exercise from last semester which i stumpled upon. However, i have some difficulties getting behind the solution i've got and i hope someone can help me.
This was the solution we were given:
$\mathscr{A} = \{Mx \mid x \in \mathbb{R}^m\} =
\operatorname{span}(v_1,…,v_m)$ with $v_1,…,v_m$ column vectors of
$M$.We have $\mathscr{A}^\bot := \{ \hat{v} \in \mathbb{R}^n \mid \langle
\hat{v},v_i \rangle = 0 \ \ \forall i=1,…,m\} = \{\hat{v} \in
\mathbb{R}^n \mid \langle \hat{v},v\rangle = 0\ \ \ \forall v \in
\mathscr{A}\}$Then it holds that $ \ \ \forall f \in \mathbb{R}^n: \exists !
v_{\mathscr{A}} \in \mathscr{A}, v_{\mathscr{A}^\bot} \in
\mathscr{A}^\bot : f = v_{\mathscr{A}} + v_{\mathscr{A}^\bot}$Therefore
$$\Vert v_{\mathscr{A}} – f \Vert = \min_{v \in \mathscr{A}} \Vert v-f
\Vert$$since $\Vert v – f \Vert = \Vert v – (v_{\mathscr{A}} +
v_{\mathscr{A}^\bot})\Vert = \Vert (v – v_{\mathscr{A}}) +
v_{\mathscr{A}^\bot} \Vert = \Vert v-v_{\mathscr{A}}\Vert + \Vert
v_{\mathscr{A}^\bot}\Vert$ becomes minimal for $v – v_{\mathscr{A}} =
0$
I have two questions i couldnt figure out on my own, even after checking all the books i currently have:
Question 1:
According to the solution of my professor
$$\Vert (v – v_{\mathscr{A}}) +
v_{\mathscr{A}^\bot} \Vert = \Vert v-v_{\mathscr{A}}\Vert + \Vert
v_{\mathscr{A}^\bot}\Vert$$ follows directly from $ (v-v_{\mathscr{A}}) \bot v_{\mathscr{A}^\bot} $
but i dont see how this applies? Shouldn't there be the triangle inequality? All i was able to figure out is that if two vectors $x,y$ are orthogonal, s.t. $x \bot y$, it holds that
$\Vert x + y \Vert^2 = \Vert x \Vert^2 + \Vert y \Vert^2$ with $\Vert \cdot \Vert$ being the euclidian norm.
Can anyone enlighten me?
And maybe an easier question is the following
Question 2:
Why exactly is $$\Vert v-v_{\mathscr{A}}\Vert + \Vert
v_{\mathscr{A}^\bot}\Vert$$
becoming minimal for $$v – v_{\mathscr{A}} =
0$$?
I really hope someone can help me getting a better understanding of the given solution. Thanks in advance!
Best Answer
I managed to get in contact with my professor and it turned out to be a typo, thus the equality i couldn't get behind was missing the exponentiation ^2