Cosine Law would be easier, but let's do it your way. Consider the small right triangle (call it $\Delta ABC$), which roughly looks like:
A----B
| /
| /
| /
|/
C
Note that $\angle ABC =59^\circ$. Thus, $AB=640\cos(59^\circ)$ and $AC=640\sin(59^\circ)$.
Now consider the large right triangle. Its two legs are $AC$ and $AB+480$. Hence, by the Pythagorean Theorem, we have:
$$x=\sqrt{(AC)^2 + (AB+480)^2}=\sqrt{(640\cos(59^\circ)+480)^2 + (640\sin(59^\circ))^2} \approx 978 \text{ miles}$$
Finally, for the angle bearing:
$$\theta=\tan^{-1}\left(\frac{AC}{AB+480}\right) = \tan^{-1}\left(\frac{640\sin(59^\circ)}{640\cos(59^\circ)+480}\right) \approx 34^\circ \text{ to get the bearing you subtract that value from 90 to get } S56^\circ W$$
I assume that the angles are measured in a clockwise direction starting from a reference direction of $0^\circ$ which is north.
Using trigonometry the problem can be reduced to find an angle in a triangle knowing two of its sides and another angle.
The water velocity is represented by the blue vector, with a magnitude of $3\, \mathrm{km/h}$. To answer the problem we need to find the direction of the black vector, whose magnitude is $35\, \mathrm{km/h}$. This means we need to find the angle $\theta=A$ of the triangle $\Delta=[A,B,C]$, knowing the sides $a=3$ and $b=35$, and the angle $B=105^\circ$. From the law of sines
$$\frac{a}{\sin A}=\frac{b}{\sin B}$$
we deduce that
$$\sin\theta=\sin A=\frac{a\sin B}{b}=\frac{3\sin 105^\circ}{35}.$$
So
$$\theta=\arcsin\left( \frac{3\sin 105^\circ}{35}\right)\approx 4.749^\circ .$$
This means that the boat should take the direction of
$$65^\circ+\theta\approx 69.75^\circ,$$
sailing with a speed of $35\, \mathrm{km/h}$. A new application of the law of sines to the side $c$ and angle $C$, yields $c=34.103\, \mathrm{km/h}$, which is the resulting speed of the boat in the port direction.
Best Answer
Here is a drawing of the problem I made (Excuse my bad drawing skills).
LH represents the lighthouse, and MB represents the motorboat.
Here, the motorboat is travelling at a $53^{\circ}$ angle from North.
Now, here is the important part to notice: The shortest distance will be the point when the perpendicular of the black line (The path which the motorboat goes) intersects the light house.
Therefore, the $\color{red}{\text{red}}$ line is the shortest distance. This length can easily be calculated using trigonometry:
$10\sin{53^{\circ}}\approx \boxed{7.986 \text{ km}}$.