Using the Law of Sines, we can write
$$\frac{a}{\sin A} = \frac{b}{\sin B} = \frac{c}{\sin C} = d$$
where $a$, $b$, $c$ are the sides, and $d$ is the circumdiameter, of the triangle. And the Law of Cosines gives us
$$\cos A = \frac{1}{2bc}(-a^2+b^2+c^2) \qquad\text{, etc.}$$
With $\sin 2 x = 2 \sin x \cos x$, we can express the determinant as
$$\left|\begin{array}{ccc}
\frac{a}{d}\frac{-a^2+b^2+c^2}{bc} & \frac{c}{d} & \frac{b}{d} \\[4pt]
\frac{c}{d} & \frac{b}{d}\frac{a^2-b^2+c^2}{ca} & \frac{a}{d} \\[4pt]
\frac{b}{d} & \frac{a}{d} & \frac{c}{d}\frac{a^2+b^2-c^2}{ab}
\end{array}\right|$$
From here, we can "factor-out" $\frac{1}{dbc}$, $\frac{1}{dca}$, $\frac{1}{dab}$ from the first, second, and third rows:
$$\frac{1}{dbc}\frac{1}{dca}\frac{1}{dab}\;\left|\begin{array}{ccc}
a(-a^2+b^2+c^2) & b c^2 & c b^2 \\[4pt]
a c^2 & b (a^2-b^2+c^2) & c a^2 \\[4pt]
ab^2 & b a^2 & c (a^2+b^2-c^2)
\end{array}\right|$$
Then, we factor-out $a$, $b$, $c$ from first, second, and third columns:
$$\frac{a b c}{d^3a^2b^2c^2}\;\left|\begin{array}{ccc}
-a^2+b^2+c^2 & c^2 & b^2 \\[4pt]
c^2 & a^2-b^2+c^2 & a^2 \\[4pt]
b^2 & a^2 & a^2+b^2-c^2
\end{array}\right|$$
Subtracting, say, the first row from the second and third gives
$$\frac{1}{d^3abc}\;\left|\begin{array}{ccc}
-a^2+b^2+c^2 & c^2 & b^2 \\[4pt]
a^2-b^2 & a^2-b^2 & a^2-b^2 \\[4pt]
a^2-c^2 & a^2-c^2 & a^2-c^2
\end{array}\right|$$
which clearly vanishes.
I take it this is a Toeplitz matrix (diagonals are constant).
Subtract row $n-1$ from row $n$, then row $n-2$ from row $n-1$, ... row $1$ from row $2$. Then subtract column $1$ from each other column.
Then add $1/n$ times the second, third, ..., last columns to the first column. The result should be upper triangular.
Best Answer
In more detail, the thing that's being suggested to get the RHS is the following reasoning: \begin{align} \begin{vmatrix}a & b & c \\ c & a & b \\ b & c & a\end{vmatrix} &= \begin{vmatrix}a+c & a+b & b+c \\ c & a & b \\ b & c & a\end{vmatrix} & \text{(Add row 2 to row 1)} \\ &= \begin{vmatrix}a+b+c & a+b+c & a+b+c \\ c& a & b \\ b & c & a\end{vmatrix} & \text{(Add row 3 to row 1)} \\ &= (a+b+c) \begin{vmatrix}1 & 1 & 1\\c & a & b\\ b & c & a\end{vmatrix} & \text{(Factor out $a+b+c$)} \\ &= (a+b+c) \left(\begin{vmatrix}a & b \\ c & a\end{vmatrix} - \begin{vmatrix}c & b \\ b & a\end{vmatrix} + \begin{vmatrix}c & a \\ b & c\end{vmatrix}\right) & \text{(Expand on $1^{\text{st}}$ row)} \\ &= (a+b+c)(a^2-bc - (ca-b^2) + c^2-ab) \\ &= (a+b+c)(a^2+b^2+c^2 - ab - bc - ca). \end{align}