You shouldn't dismiss Karo's graph. Drawing a graph to get a feel for the equation (when you don't know how to proceed) is most helpful. And this isn't nonsense, in fact the graph is the key for the solution, as it makes it clear that we can consider only $x$ for which $-2 \leq x \leq 2$. This can be conveniently rewritten as $-1\leq\frac{x}{2}\leq1$.
This inequality reminds us immediately of the one that $\sin a$ and $\cos a$ also satisfy, and indeed we can assume WLOG $\frac{x}{2}=\sin a$, for some angle $a$ in radians of course. This makes $x=2 \sin a$, so let's substitute this in the equation.
$$\begin{align}
8\sin^3 a-6\sin a&=\sqrt{2(\sin a+1)}\\
-2(-4\sin^3a+3\sin a)&=\sqrt{2(\sin a+1)}\\
-2\sin3a&=\sqrt{2(\sin a+1)}
\end{align}$$
It helped that we could factor it into $\sin 3a$, but then again we can't get rid of the square root in a nice way. This is because $\sin^2a$ is given in terms of $\cos 2a$, and not $\sin 2a$, so we can't turn the radicand into a nice square of a sine. On the other hand, $\cos^2 a$ can be expressed in terms of a cosine (indeed, we have the trigonometric identity $\cos^2a=\frac{1+\cos2a}{2}$). So we are lead to believe that the substituition $x=2\cos a$ is much more fortunate. It's worth a shot:
$$\begin{align}
8\cos^3 a-6\cos a&=\sqrt{2(\cos a+1)}\\
2(4\cos^3a-3\cos a)&=\sqrt{4\frac{(\cos a+1)}{2}}\\
2\cos3a&=\sqrt{2^2\cos^2\frac{a}{2}}\\
\cos3a &=\cos\frac{a}{2}
\end{align}$$
The equation has been successfully trivialized. We should note that imposing $0\leq a\leq\pi$, $\cos a$ will still assume all of the values it possibly could, so we impose this restriction on $a$.
Therefore, we can have $3a=\frac{a}{2}\Rightarrow a=0$, which gives us the solution $x=2\cos0=2$, or we can have the other, non-trivial solutions:
$$3a=\frac{a}{2}+2\pi\Rightarrow a=\frac{4\pi}{5}$$
which gives us $x=2\cos\frac{4\pi}{5}=-2\cos\frac{\pi}{5}$. If $0\leq a\leq\pi$, then $0\leq 3a \leq 3\pi\lt4\pi$, so we don't need to look the case $3a=\frac{a}{2}+4\pi$ or higher, and we just need to consider the last case:
$$3a=2\pi-\frac{a}{2}\Rightarrow a=\frac{4\pi}{7}$$
Which gives us the last solution, $x=2\cos\frac{4\pi}{7}=-2\cos\frac{3\pi}{7}$.
Note: This solution was inspired by the one brazilian mathematician Nicolau C. Saldanha presented in a lesson. Here's the link: http://y2u.be/jFFdOSsVGgg (someone who's fluent in Spanish shouldn't have trouble understanding the explanations). It is also shown that with some simple manipulations on the regular pentagon, one can arrive at $\cos\frac{\pi}{5}=\frac{1+\sqrt{5}}{4}$, so the second solution can be written as $x=-\frac{1+\sqrt{5}}{2}$ (which justifies the quadratic factor $x^2-x-1$ of the 6th degree polynomial you get when you square both sides, which obviously contains some extraneous roots).
I'm not sure there's an easy way to turn $\cos\frac{3\pi}{7}$ into one algebraic term (if there is one at all), so we just leave it that way. The solution set is $$x \in \left. \left \{ 2,-\frac{1+\sqrt{5}}{2},-2\cos\frac{3\pi}{7} \right. \right \}$$
Best Answer
Simplify as $$4x^2[(x+1)^2+1]=5(x+1)^2 \implies 4x^4+8x^3+3x^2-10x-5=0$$ Denoting LHS as $f(x)$, note that $f(1)=0 ~\& ~f(-1/2)=0.$ This means that by remainder theorem $$f(x) ~\text{is divisible by}~ (2x+1)(x-1)=2x^2-x-1=g(x)$$ dividing $f(x)$ by $g(x)$ we see that $$f(x)=(2x^2-x-1)(2x^2+5x+5)$$ By solving $2x^2+5x+5=0$ we get $$x=\frac{-5\pm \sqrt{-15}}{4}=\frac{-5\pm i\sqrt{15}}{4}.$$
Finally, the equation has 4 roots $$x=-1/2,1,\frac{-5\pm i\sqrt{15}}{4}.$$