Find all least squares solutions of A x = b, where A =
\begin{bmatrix}
1 & 3 \\[0.3em]
-2 &-6 \\[0.3em]
3 & 9
\end{bmatrix}
and b = \begin{bmatrix}
1 \\[0.3em]
0 \\[0.3em]
1
\end{bmatrix}
and confirm that all
the solutions have the same error vector (and hence the same least squares error).
Compute the least-squares error.
The system that corresponds to the reduced row echelon form of the augmented matrix is $x_1 + 3x_2 =0, 0=1, 0=0 $ since the second equation cannot be solved the system is inconsistent
I'm not sure how to calculate the least squares solutions of A x = b because the inverse of $A^TA$ does not exist and so I can't solve the normal equations $A^TAx=A^Tb$ for x
Best Answer
Even though $A^TA$ is singular, you can solve the normal equations. Note that, from a theoretical perspective, $A^Tb$ is in the column space (image) of $A^T$, and the column space of $A^TA$ is the same subspace — again, for emphasis, always.