Gram-Schmidt Procedure from “Linear Algebra Done Right”

abstract-algebragram-schmidtlinear algebraproof-verification

The following content is from "Linear Algebra Done Right" book by Sheldon Axler, 6.31.

There was a part of the proof what I don't understand is that

$\begin{align*}
\left\langle e_j, e_k\right\rangle &= \left\langle\frac{v_j – \left\langle v_j, e_1\right\rangle e_1 – …… – \left\langle v_j, e_{j-1}\right\rangle e_{j-1}}{||v_j – \left\langle v_j, e_1\right\rangle e_1 – … – \left\langle v_j, e_{j-1}\right\rangle e_{j-1}||}, e_k\right\rangle\\ &= \frac{\left\langle v_j, e_k\right\rangle – \left\langle v_j, e_k\right\rangle }{||v_j – \left\langle v_j, e_1\right\rangle e_1 – … – \left\langle v_j, e_{j-1}\right\rangle e_{j-1}||}\\
&= 0
\end{align*}$

How did he got from the first equation into the second equation?

Best Answer

The inner product is bi-linear. This basically means that it can distribute across its inputs. For the purpose of my answer I am going to assume you are working in a real vector space; if the space is complex then the modifications necessary are very minor.

The bi-linearity of the inner product means that the following holds.

$$ \langle a V_1 + b V_2, c U_1 + d U_2 \rangle = ac \langle V_1, U_2\rangle + ad \langle V_2, U_1 \rangle + bc \langle V_2, U_1 \rangle + bd \langle V_2, U_2 \rangle$$

If your vector space is complex, then some of these coefficients will be complex conjugated.

Lets apply this identity to your problem.

$$\langle e_j, e_k \rangle = \Big\langle \frac{v_j - \langle v_j, e_1 \rangle e_1 - \dots - \langle v_j, e_{j-1}\rangle e_{j-1}}{\| v_j - \langle v_j, e_1\rangle e_1 - \dots - \langle v_j, e_{j-1}\rangle e_{j-1} \|}, e_k\Big\rangle$$

$$= \frac{\Big\langle v_j - \langle v_j, e_1 \rangle e_1 - \dots - \langle v_j, e_{j-1}\rangle e_{j-1}, e_k\Big\rangle}{\| v_j - \langle v_j, e_1\rangle e_1 - \dots - \langle v_j, e_{j-1}\rangle e_{j-1} \|}$$

$$= \frac{\Big\langle v_j, e_k\Big\rangle - \langle v_j, e_1 \rangle \Big\langle e_1, e_k\Big\rangle - \dots - \langle v_j, e_{k}\rangle \Big\langle e_{k}, e_k\Big\rangle \dots- \langle v_j, e_{j-1}\rangle \Big\langle e_{j-1}, e_k\Big\rangle}{\| v_j - \langle v_j, e_1\rangle e_1 - \dots - \langle v_j, e_{j-1}\rangle e_{j-1} \|}$$

By construction the basis vectors $e_1 \dots e_k \dots e_{j-1}$ are ortho-normal. This means that $\langle e_i, e_k \rangle = 0$ when $i \neq k$ and $\langle e_k, e_k \rangle = 1$ for $1\leq i \leq j-1$.

$$= \frac{\Big\langle v_j, e_k\Big\rangle - \langle v_j, e_1 \rangle 0 - \dots - \langle v_j, e_{k}\rangle 1 \dots- \langle v_j, e_{j-1}\rangle 0}{\| v_j - \langle v_j, e_1\rangle e_1 - \dots - \langle v_j, e_{j-1}\rangle e_{j-1} \|}$$

$$= \frac{\Big\langle v_j, e_k\Big\rangle - 0 - \dots - \langle v_j, e_{k}\rangle \dots- 0}{\| v_j - \langle v_j, e_1\rangle e_1 - \dots - \langle v_j, e_{j-1}\rangle e_{j-1} \|}$$

$$= \frac{\Big\langle v_j, e_k\Big\rangle - \langle v_j, e_{k}\rangle }{\| v_j - \langle v_j, e_1\rangle e_1 - \dots - \langle v_j, e_{j-1}\rangle e_{j-1} \|}$$

$$= \frac{0 }{\| v_j - \langle v_j, e_1\rangle e_1 - \dots - \langle v_j, e_{j-1}\rangle e_{j-1} \|}$$

$$= 0$$