I am studying the book Linear Algebra from Hoffman and Kunze. The authors make the following comment on page 281.
Although it is of limited practical use for computations, it is
interesting to note that the Gram-Schmidt process may also be used to
test for linear dependence.
I have two question on that comment:
1) Why should I study the Gram-Schmidt orthogonalization process?
2) Is there an example where the Gram-Schmidt orthogonalization process makes easier to prove that a set of vectors is linearly dependent instead of use another method? I've never proved that a subset of vectors was linearly independent by using the Gram-Schmidt orthogonalization process.
Maybe I did not understand what they are saying.
Best Answer
The Gram-Schmidt orthogonalization process is a great thing to learn about because the idea behind it shows up again and again. There are many practical algorithms that use what is essentially a Gram-Schmidt procedure. For example, suppose you have a set $\{v_1, \dots, v_n\}$ of linearly independent vectors (or functions), and you want to approximate another vector (or function) $w$ as a linear combination of the vectors in your set. This can be done by linear regression -- i.e. project $w$ onto the space spanned by the set. An alternative method, which is useful when you have a huge collection of functions in your set, is called "matching pursuit." Here you project $w$ onto the space spanned by the vector $v_i$ that is most correlated with $w$, subtract off that component, then project what remains onto the next most correlated vector from your set, subtract, etc. This process of projecting-subtracting, projecting-subtracting,... is just like Gram-Schmidt, and it is the basic principal behind many practical algorithms for time-frequency decompositions, time-scale (wavelet) decompositions, etc. In other words, there are many "Gram-Schmidt-like" procedures, so you would do well to learn the original!