I think that the method of Lagrange multipliers is the easiest way to
solve my question, but how can I find the Lagrangian function?
As shown by other answers and in note 1 there are easier ways to find the shortest distance, but here is a detailed solution using the method of Lagrange multipliers. You need to find the minimum of the distance function
$$\begin{equation}
d(x,y,z)=\sqrt{x^{2}+y^{2}+(z-1)^{2}} \tag{1a}
\end{equation}$$
subject to the constraint given by the surface equation $z=f(x,y)=\frac32(x^2+y^2)$
$$\begin{equation}
g(x,y,z)=z-\frac{3}{2}\left( x^{2}+y^{2}\right) =0. \tag{2}
\end{equation}$$
Since $\sqrt{x^{2}+y^{2}+(z-1)^{2}}$ increases with $x^{2}+y^{2}+(z-1)^{2}$ you can simplify the
computations if you find the minimum of
$$\begin{equation}
[d(x,y,z)]^2=x^{2}+y^{2}+(z-1)^{2} \tag{1b}
\end{equation}$$
subject to the same constraint $(2)$. The Lagrangian function is then
defined by
$$\begin{eqnarray}
L\left( x,y,z,\lambda \right) &=&[d(x,y,z)]^2+\lambda g(x,y,z) \\
L\left( x,y,z,\lambda \right) &=&x^{2}+y^{2}+(z-1)^{2}+\lambda \left( z-
\frac{3}{2}\left( x^{2}+y^{2}\right) \right), \tag{3}
\end{eqnarray}$$
where $\lambda $ is the Lagrange multiplier. By this method you need to
solve the following system
$$\begin{equation}
\left\{ \frac{\partial L}{\partial x}=0,\frac{\partial L}{\partial y}=0,
\frac{\partial L}{\partial z}=0,\frac{\partial L}{\partial \lambda }
=0,\right. \tag{4}
\end{equation}$$
which results in
$$\begin{eqnarray}
\left\{
\begin{array}{c}
2x+3\lambda x=0 \\
2y+3\lambda y=0 \\
2z-2-\lambda =0 \\
-z+\frac{3}{2}\left( x^{2}+y^{2}\right) =0
\end{array}
\right. &\Leftrightarrow &\left\{
\begin{array}{c}
x=0\vee 2+3\lambda =0 \\
y=0\vee 2+3\lambda =0 \\
2z-2-\lambda =0 \\
-z+\frac{3}{2}\left( x^{2}+y^{2}\right) =0
\end{array}
\right. \\
&\Leftrightarrow &\left\{
\begin{array}{c}
x=0 \\
y=0 \\
\lambda =2 \\
z=0
\end{array}
\right. \vee \left\{
\begin{array}{c}
\lambda =-2/3 \\
z=2/3 \\
x^{2}+y^{2}=4/9
\end{array}
\right. \tag{5}
\end{eqnarray}$$
For $x=y=x=0$ we get $d(0,0,0)=1$. And for $x^2+y^2=4/9,z=2/3$ we get the minimum distance subject to the given conditions
$$\begin{equation}
\underset{g(x,y,z)=0}\min d(x,y,z)=\sqrt{\frac{4}{9}+(\frac{2}{3}-1)^{2}}=\frac{1}{3}\sqrt{5}. \tag{6}
\end{equation}$$
It is attained on the intersection of the surface $z=\frac{3}{2}\left(
x^{2}+y^{2}\right) $ with the vertical cylinder $x^{2}+y^{2}=\frac{4}{9}$ or equivalently with the horizontal plane $z=\frac{2}{3}$.
$$\text{Plane }z=\frac{2}{3} \text{(blue) and surface }z=\frac{3}{2}\left(
x^{2}+y^{2}\right) $$
Notes.
- As the solution depends only on the sum $r^{2}=x^{2}+y^{2}$ we could just find
$$\begin{equation}
\min [d(r)]^2=r^{2}+(\frac{3}{2}r^{2}-1)^{2} \tag{7}
\end{equation}$$
and then find $d(r)=\sqrt{[d(r)]^2}$ at the minimum.
- The surface $z=\frac{3}{2}\left(
x^{2}+y^{2}\right) $ is a surface of revolution around the $z$ axis.
Best Answer
First, draw a picture. Let $Q$ be the point $(a,b,c)$ and let $\theta$ be the angle between the vectors $\vec{v}$ and $\vec{PQ}$. Then, the point $R$ on the line $\ell$ which is closest to $Q$ is the point such that $\angle QRP = 90^{\circ}$, i.e. $QR$ is perpendicular to $\ell$. Then, the shortest distance from $Q$ to $\ell$ is $\|\vec{QR}\| = \|\vec{PQ}\|\sin\theta$. Using the formula $\|\vec{PQ} \times v\| = \|\vec{PQ}\|\|\vec{v}\|\sin\theta$, we get $\|\vec{QR}\| = \|\vec{PQ}\|\sin\theta = \dfrac{\|\vec{PQ} \times v\| }{\|v\|}$.
Alternatively, Google "distance between a point and a line" and click on the Wikipedia article.