Question. Let $a,b,c>0: abc=a+b+c+2.$ Prove that$$2\left(\sqrt{ab-1}+\sqrt{bc-1}+\sqrt{ca-1}\right)\le \left(\sqrt{a}+\sqrt{b}+\sqrt{c}\right)\sqrt{\sqrt{ab}+\sqrt{bc}+\sqrt{ca}}.$$I am looking for a nice proof by hand, for which there is a possibility to find this proof during a competition.
Equality holds at $a=b=c=2.$
I've tried to use the well-known substitution without success.
Indeed, by the given condition we easily get $$\frac{1}{a+1}+\frac{1}{b+1}+\frac{1}{c+1}=1.$$
Now, if we set $a=\dfrac{y+z}{x};b=\dfrac{x+z}{y}$ then $c=\dfrac{y+x}{z}.$ Replace the set to OP, it's very complicated.
Hope to see some ideas. All contributions are welcome.
Best Answer
Not sure where this will lead but let's give it a go!
Let $$A^2 = \frac{1}{a+1}, \quad B^2 = \frac{1}{b+1}, \quad C^2 = \frac{1}{c+1}$$ So that $A^2 + B^2 + C^2 = 1$. (like OP mentioned)
Then we simply get $$a = \frac{1-A^2}{A^2}, \quad b = \frac{1-B^2}{B^2}, \quad c = \frac{1-C^2}{C^2}$$ so $$ab = \left(\frac{1}{A^2}-1\right)\left(\frac{1}{B^2}-1\right)$$ $$ = \frac{1}{A^2B^2}-\frac{1}{A^2}-\frac{1}{B^2}+1$$ $$\sqrt{ab-1} = \sqrt{\frac{1-(A^2+B^2)}{A^2B^2}} = \frac{\sqrt{1-(A^2+B^2)}}{AB}$$
Plugging all of this in and squaring both sides we get $$4\left(\frac{\sqrt{1-(A^2+B^2)}}{AB}+\frac{\sqrt{1-(B^2+C^2)}}{BC}+\frac{\sqrt{1-(A^2+C^2)}}{AC}\right)^2 \leq \,\,\,...$$
$$4\left(\frac{C}{AB}+\frac{A}{BC}+\frac{B}{AC}\right)^2 \leq \left(\frac{\sqrt{1-A^2}}{A}+\frac{\sqrt{1-B^2}}{B}+\frac{\sqrt{1-C^2}}{C}\right)^2\left(\frac{\sqrt{C^2+A^2B^2}}{AB}+\frac{\sqrt{A^2+B^2C^2}}{BC}+\frac{\sqrt{B^2+A^2C^2}}{AC}\right)$$ We can add the terms in the brackets and after simplifying get $$\frac{4}{(ABC)^2} \leq \frac{\left(BC\sqrt{1-A^2}+AC\sqrt{1-B^2}+AB\sqrt{1-C^2}\right)^2}{(ABC)^2}\left(\frac{C\sqrt{C^2+A^2B^2}+B\sqrt{B^2+A^2C^2}+A\sqrt{A^2+B^2C^2}}{ABC}\right)$$ simplifying again $$4ABC \leq \left(BC\sqrt{1-A^2}+AC\sqrt{1-B^2}+AB\sqrt{1-C^2}\right)^2\left(C\sqrt{C^2+A^2B^2}+B\sqrt{B^2+A^2C^2}+A\sqrt{A^2+B^2C^2}\right)$$
I wasn't sure what the most efficient way from here on out was. I had made a few attempts but they didn't seem to simplify anything any further, so the following is just something that I think might help anyone else also trying to solve this.
I don't think there is much point in showing attempts, and thought it would be better to show everyone how to get back to an inequality in $a, b, c$ alone while I keep attempting at the final solution.
To get it back into a form in terms of $a,b,c$, we can divide both sides by $(ABC)^3$ to obtain
$$\frac{4}{(ABC)^2} \leq \left(\frac{\sqrt{1-A^2}}{A}+\frac{\sqrt{1-B^2}}{B}+\frac{\sqrt{1-B^2}}{B}\right)^2\left(\sqrt{\frac{C^2}{A^2B^2}+1}+\sqrt{\frac{B^2}{A^2C^2}+1}+\sqrt{\frac{A^2}{B^2C^2}+1}\right)$$
Now we can re-write everything in terms of $a, b, c$ again using the identities, and we arrive at
$$4(a+1)(b+1)(c+1) \leq \left(\sqrt{a}+\sqrt{b}+\sqrt{c}\right)^2\left(\sqrt{ab}+\sqrt{ac}+\sqrt{bc}\right)$$
We can simplify the RHS slightly to get
$$4abc+12 \leq \left(\sqrt{a}+\sqrt{b}+\sqrt{c}\right)^2\left(\sqrt{ab}+\sqrt{ac}+\sqrt{bc}\right)$$
Again I would just like to clarify that this is incomplete! If anyone spots a mistake or inconsistency please let me know.
Edit: The inequality has been solved by a different answer, and I can't seem to get to a solution using this method, so I'll say for now unless I find a way to solve it I probably will not be updating\adding to my approach here anytime soon.