I am reviewing Example 1 from Chapter 6, Section 4 (Least-Squares Approximation and Orthogonal Projection Matrices) in "Elementary Linear Algebra – A Matrix Approach 2nd Edition [ISBN] 978-0-13-187141-0"
In this example, they found a solution (2×1 matrix) of the normal equation:
[a0]
= (((C^T)*C)^-1)(C^T)*y
[a1]
Given:
C is a 5×2 matrix; y is a 5×1 matrix
C = [1 2.60] y = [2.00]
[1 2.72] [2.10]
[1 2.75] [2.10]
[1 2.67] [2.03]
[1 2.68] [2.04]
*((C^T)C) is a 2×2 matrix
((C^T)*C) = [5.0000 13.4200]
[13.4200 36.0322]
(C^T)*y is a 2×1 matrix
(C^T)*y = [10.2700]
[27.5743]
The answer was:
[a0] = [0.056]
[a1] = [0.745]
To solve this I think they had to use the formula I listed at the very top, but they did not show work for ((C^T)*C)^-1 (which I guess is the inverse). If someone can please explain with full details of how they solved this normal equation.
I at least understand the given equations I posted, but I don't know why they didn't show the steps of ((C^T)*C)^-1 and how exactly they arrived to:
[a0] = [0.056]
[a1] = [0.745]
y = 0.056 + 0.745x
Best Answer
The normal equation is $$C^tCx=C^ty$$ which is to say, $$\pmatrix{5&13.42\cr13.42&36.0322\cr}\pmatrix{a_0\cr a_1}=\pmatrix{10.27\cr27.5743\cr}$$ This is just solving two equations in two unknowns, and you can solve such a system by any method you know (and surely you know how to solve two equations in two unknowns).
Now, one way to solve it is to multiply both sides by $(C^tC)^{-1}$ which is, indeed, the multiplicative inverse of $C^tC$; you get the solution $x=(C^tC)^{-1}C^ty$. So your question is, how do you find the inverse of a matrix.
For $2\times2$ matrices, there is a very simple answer: $${\rm The\ inverse\ of\ }\pmatrix{a&b\cr c&d\cr}{\rm\ is\ }(ad-bc)^{-1}\pmatrix{d&-b\cr-c&a}$$
For bigger matrices, there is a simple procedure, involving row reduction.
But surely all of this is in some earlier chapter of the textbook you're using?