Note that $\theta = \cos^{-1}x$ function, by definition, has domain $x\in [-1,1]$ and range in $\theta \in [0,\pi]$, such that
$$\theta =\cos^{-1}x \iff \cos \theta = x$$
we also have
$$\cos (\pi -\theta) = -x$$
and therefore
$$\cos^{-1}(-x)=\pi-\theta \iff \cos (\pi-\theta) = -x$$
that is
$$\cos^{-1}x + \cos^{-1}(-x)=\theta+\pi-\theta= \pi$$
which leads to $\forall x\in [-1,1]$
$$\cos^{-1}x + \cos^{-1}(-x)=\pi$$
In this case we have $\sin \frac{4\pi}{3} = -\frac{\sqrt 3}2$ and $\cos^{-1} \left(\frac{\sqrt 3}2\right)=\frac \pi 6$ then
$$\cos^{-1} \left(\sin \frac{4\pi}{3}\right)=\cos^{-1} \left(-\frac{\sqrt 3}2\right)= \pi-\cos^{-1} \left(\frac{\sqrt 3}2\right)$$
For the first point, by symmetry, from the unit circle, we have that
$$\sin \theta = \cos \left(\frac \pi 2-\theta\right)$$
therefore
$$\sin \frac{4\pi}{3}= \cos \left(-\frac {5\pi} 6\right)=-\cos \left(\frac {\pi} 6\right)=-\frac{\sqrt 3}2$$
or also
$$\sin \theta = \sin (\pi -\theta) \implies \sin \frac{4\pi}{3}= \sin \left(-\frac{\pi}{3}\right)=-\sin \left(\frac{\pi}{3}\right)=-\frac{\sqrt 3}2$$
Best Answer
For trigonometry problems, you should preferably consider the trigonometric circle, not triangles.
$$\cos^2u=1-\Bigl(\frac{-8}{17}\Bigr)^2=\frac{225}{289}=\Bigl(\frac{15}{17}\Bigr)^2.$$ Now, on the interval $\Bigl[\pi,\frac{3\pi}2\Bigr]$, $\cos u \le 0$, so $\cos u=-\frac{15}{17}$, and $$\sin 2u=2\cdot\frac{-8}{17}\cdot\frac{-15}{17}=\frac{240}{289}.$$