Finding point on a plane closest to a point in $\mathbb{R}^n$ using least squares method

Question

Suppose S is a 2d plane in $\mathbb{R}^3$ s.t. it is the set of all vectors in $\mathbb{R}^3$ with $ax_1+bx_2=0$ (a,b not equal to 0). Let $b=(x_1,x_2,x_3)^T$ be any vector in $\mathbb{R}^3$. How can I use least squares fitting to find the point in S that's closest to b? I understand it will be the projection, but how does least squares come into play? Do I have to use formula $x_{min}=(A^TA)^{-1}A^Tb$?

Answer 1

Let us first consider the case when it is required to find the point on the $$\tag{1} P:\;ax+by+cz=d $$ closest to the origin. The coordinates of the point $Q$ on the plane described by (1) nearest to the origin are equal to the minimal norm solution of (1): $$\tag{2} \left(\begin{array}{c} x_Q\\y_Q\\z_Q \end{array}\right)= \left(\begin{array}{ccc} a& b &c \end{array}\right)^+\cdot d $$ Now consider the case when it is required to find the point on the plane (1) closest to some arbitrary point $M(x_M,y_M,z_M)$. After the change of variables $$ x'=x-x_M,\quad y'=y-y_M,\quad z'= z-z_M $$ $M$ tranforms into the origin and (1) transforms into $$ a(x'+x_M)+b(y'+y_M)+c(z'+z_M)=d $$ or $$ ax'+by'+cz'=d-ax_M-by_M-cz_M. $$ According to (2), we obtain $$ \left(\begin{array}{c} x'_Q\\y'_Q\\z'_Q \end{array}\right)= \left(\begin{array}{ccc} a& b &c \end{array}\right)^+\cdot (d-ax_M-by_M-cz_M) $$ and $$ x_Q= x'_Q+x_M,\quad y_Q= y'_Q+y_M,\quad z_Q= z'_Q+z_M. $$ Since $\left(\begin{array}{ccc} a& b &c \end{array}\right)$ has linearly independent rows, $$ \left(\begin{array}{ccc} a& b &c \end{array}\right)^+= \left(\begin{array}{ccc} a& b &c \end{array}\right)^T (\left(\begin{array}{ccc} a& b &c \end{array}\right)\left(\begin{array}{ccc} a& b &c \end{array}\right)^T)^{-1} = \frac1{a^2+b^2+c^2} \left(\begin{array}{c} a\\b\\c \end{array}\right), $$ thus, $$ \left(\begin{array}{c} x_Q\\y_Q\\z_Q \end{array}\right)= \frac{d-ax_M-by_M-cz_M}{a^2+b^2+c^2} \left(\begin{array}{c} a\\b\\c \end{array}\right)+ \left(\begin{array}{c} x_M\\y_M\\z_M \end{array}\right). $$