I'm trying to understand gram-schmidt orthogonalization process. And it says $p_1,p_2…p_N$ are linearly independent vectors and $q_1,q_2…q_N$ are orthonormal vectors created from them.
And also there are some equations like in the below:
$q_1$ is unit vector so it's length is 1 hence $q_1 = p_1/ |p_1|$ and also q1.q1 = 1 because it is unit. But there is an equation that I didn't understand. What means the $$q_2 = a*p_1 + b*p_2 = alpha*q_1 + beta*p_2$$ How was this equality achieved? Is a and b are coefficient/number or angle? What is the geometric explanation for this?
Can anybody explain this equation?
Best Answer
Recall your statement :
Indeed, since this is the ultimate goal of the Gram-Schmidt process! To construct a set of orthonormal vectors from your original set of vectors as I highlighted your comment in blue. These two sets should share most of the vector properties.
[Edit] : To make things more clear, I will for the sake of simplicity reduce the set of LI vectors $V$ to just two vectors. Define : $$ V:=\{v_{1},v_{2}\} $$ This set forms a basis in $\mathbb{R}^{2}$. We shall construct the desired set : $$ Q:=\{q_{1},q_{2}\} $$ such that the elements of $Q$ are the orthonormal vectors of $V$ and note that : $$ \operatorname{span}(V)=\operatorname{span}(Q) $$ The GSP begins by defining the first element of $Q$ that is $q_{1}$ : $$ w_{1}:=v_{1}\implies q_{1}:=\frac{w_{1}}{\|w_{1}\|_{2}}=\frac{v_{1}}{\|v_{1}\|_{2}} $$ Next, let $w_{2}:=v_{2}-\operatorname{proj}_{v_{1}}v_{2}$. Geometrically, the projection is more of a "copy" of $v_{2}$ but by forcing it to be aligned in the same direction of $v_{1}$. It turns out that $w_{2}$ is formed by vector addition of $v_{2}$ and $-\operatorname{proj}_{v_{1}}v_{2}$. Therefore, what you get essentially is : \begin{align} w_{2}&:=v_{2}-\operatorname{proj}_{v_{1}}v_{2}\\ &=v_{2}-\frac{v_{1}^{\intercal}v_{2}}{v_{1}^{\intercal}v_{1}}v_{1}\\ &=v_{2}-\frac{v_{1}^{\intercal}v_{2}}{\|v_{1}\|_{2}^{2}}v_{1}\\ &=v_{2}-(q_{1}^{\intercal}v_{2})q_{1} \end{align} and what remains is normalizing the vector : $$ q_{2}:=\frac{w_{2}}{\|w_{2}\|_{2}}=\frac{v_{2}-(q_{1}^{\intercal}v_{2})q_{1}}{\|v_{2}-(q_{1}^{\intercal}v_{2})q_{1}\|_{2}} $$ Now for what you are concerned about are the coefficients. Well note that : $$ v_{1}=\|v_{1}\|_{2}q_{1}=c_{1}q_{1} $$ and \begin{align} v_{2}&=\|v_{2}-(q_{1}^{\intercal}v_{2})q_{1}\|_{2}q_{2}+(q_{1}^{\intercal}v_{2})q_{1}\\ &=c_{2}q_{2}+(q_{1}^{\intercal}v_{2})q_{1} \end{align} as you can see $q_{1},v_{1}$ and $q_{2},v_{2}$ are all related through $c_{1}$ and $c_{2}$ and this affirms our early statement that the span of $V$ and $Q$ must be the same. In fact, we can write : $$ \mathbf{V}=\begin{bmatrix}v_{1}&v_{2}\end{bmatrix}_{2\times 2}=\begin{bmatrix}q_{1}&q_{2}\end{bmatrix}_{2\times 2}\begin{bmatrix}c_{1}&q_{1}^{\intercal}v_{2}\\ 0&c_{2}\end{bmatrix}_{2\times 2}=\mathbf{QR} $$ where $Q^{\intercal}Q=I$ (i.e., orthonormal matrix) and $R$ is an upper triangular $\color{blue}{\text{invertible}}$ matrix (it is clear why this matrix of coefficients must be invertible).
From this you can generalize this for all $n$ vectors of $V$ in $\mathbb{R}^{n}$.