So I figured answer:
(iii) Using the distance formula, the shortest distance D from B to π1 is given by:
D = |ax + by + cz + d| / |n| where n is the normal to plane; a, b, c, d are the coefficients of the equation of the plane; x, y, z are the coordinates of the point from the plane.
π1: 8i + j - 14k = 75
B: 2i + 3j + 5k
D = |(8)(2) + (1)(3) + (-14)(5) + (-75)| / sqrt(64+1+196)
D = |-126| / sqrt(261)
D = 126/16.155 = 7.799 ~= 7.80
You can use a formula, although I think it's not too difficult to just go through the steps. I would draw a picture first:
![enter image description here](https://i.stack.imgur.com/YJyxF.png)
You are given that $\vec{p} = (1,0,1)$ and you already found $\vec{m} = (1, -2, 4)$ and $\vec{l}_0 = (1,2,-1)$. Now it's a matter of writing an expression for $\vec{l}(t) - \vec{p}_0$:
\begin{align}
\vec{l}(t) - \vec{p}_0 =&\ (\ (t + 1) - 1\ ,\ (-2t + 2) - 0\ ,\ (4t - 1) - 1\ )\\
=&\ (\ t\ ,\ -2t + 2\ ,\ 4t - 2\ )
\end{align}
Now you dot this with the original slope of the line (recall that $\vec{l}(t) - \vec{p}_0$ is the slope of the line segment connecting the point and the line). When this dot product equals zero, you have found $t_0$ and thus $\vec{x}_0$:
\begin{align}
\vec{m} \circ (\vec{l}(t) - \vec{p}_0) =&\ (1,-2,4)\circ(\ t\ ,\ -2t + 2\ ,\ 4t - 2\ ) \\
=&\ t + 4t - 4 + 16t - 8 \\
=&\ 21t - 12
\end{align}
Setting this to $0$ gives that $21t_0 - 12 = 0 \rightarrow t_0 = \frac{4}{7}$. This gives the point $\vec{x}_0$ as:
\begin{align}
\vec{x}_0 =&\ \vec{l}(t_0) = (\ \frac{4}{7} + 1\ ,\ -\frac{8}{7} + 2\ ,\ \frac{16}{7} - 1\ ) \\
=&\ \frac{1}{7}(11, 6, 9)
\end{align}
So finally the distance would be the distance from $\vec{p}_0$ to $\vec{x}_0$:
\begin{align}
d =&\ \sqrt{\left(\frac{11}{7} - 1\right)^2 + \left(\frac{6}{7} - 0\right)^2 + \left(\frac{9}{7} - 1\right)^2}\\
=&\ \sqrt{\left(\frac{4}{7}\right)^2 + \left(\frac{6}{7}\right)^2 + \left(\frac{2}{7}\right)^2} \\
=&\ \frac{1}{7}\sqrt{4^2 + 6^2 + 2^2}\\
=&\ \frac{1}{7}\sqrt{56} \\
=&\ \frac{2}{7}\sqrt{14}
\end{align}
...or perhaps $\sqrt{\frac{8}{7}}$ is more appealing.
Extra Info
There's no need to worry about whether or not my 2D picture is really representative--it is. No matter how high the dimensions of the problem, the problem itself can always be mapped to exactly 2 dimensions unless the point is on the line--then it's a 1 dimensional problem--which of course we can represent in 2 dimensions just as we can represent this 2 dimensional problem in much higher ones.
Best Answer
Note that for any point $\vec{x}$ on the plane the dot product $\vec{n}\cdot\vec{x}$ must be constant:
$$2x+3y+6z=const$$
You can determine this constant using the given point $A$:
$$2\cdot1+3\cdot5+6\cdot3=35$$
So we get for the equation of the plane
$$2x+3y+6z=35\tag{1}$$
The distance from the origin is the length of the projection of a vector from the origin to any point on the plane onto the normal vector. This is simply the constant on the right hand side of (1) divided by the length of the normal vector:
$$\textrm{distance}=\frac{35}{|\vec{n}|}=\frac{35}{\sqrt{2^2+3^2+6^2}}=\frac{35}{7}=5$$