$a,b,c\ge 0,a+b+c=3.$ Prove:
$$\frac{ab^2}{1+2b^2+c^2}+\frac{bc^2}{1+2c^2+a^2}+\frac{ca^2}{1+2a^2+b^2} \le \frac{3}{4}$$
This problem was found in this post. As you can see, no one in that post gave a correct proof but someone pointed out that this inequality might be a problem from Mathematical Reflections. However, the archive of the journal can't be downloaded. Therefore, we must find our own solution.
I tried it myself surely. And because all variables show up in a single fraction, we are hard to use technique like tangent line. I tried to homogenise it, and it turned out to be
(in case you didn't understand, this triangle denotes the coefficients of ever term of the polynomial. From the left-top is coefficient of $a^7$, and right-top is that of $b^7$, and the bottom is that of $c^7$. And for instance, the "$762$" on the second term of the second line denotes the coefficient of $a^5bc$)
which is almost impossilbe to create a proof directly from it by hand.(Sure, since expanding is not always a perfect way)
Can you come up with a solution with wit?
Best Answer
By AM-GM $$\sum_{cyc}\frac{ab^2}{1+2b^2+c^2}\leq\sum_{cyc}\frac{ab^2}{b^2+c^2+2b}=\sum_{cyc}\frac{3ab^2}{5b^2+3c^2+2ab+2bc}$$ and it's enough to prove that: $$\sum_{cyc}\frac{3ab^2}{5b^2+3c^2+2ab+2bc}\leq\frac{3}{4}$$ or $$\sum_{cyc}\frac{3ab^2}{5b^2+3c^2+2ab+2bc}\leq\frac{a+b+c}{12}$$ or $$\sum_{cyc}(30a^6b-63a^5b^2+45a^5c^2+63a^4b^3-15a^4c^3+20a^5bc-103a^4b^2c+95a^4c^2b+108a^3b^3c-180a^3b^2c^2)\geq0.$$ Now, since $$\sum_{cyc}(26a^6b-63a^5b^2+2a^5c^2+50a^4b^3-15a^4c^3)=$$ $$=\sum_{cyc}a^2b(26a^4-63a^3b+50a^2b^2-15ab^3+2b^4)=$$ $$=\sum_{cyc}a^2b(a-b)^2(26a^2-11ab+2b^2)\geq0,$$ it's enough to prove that:$$\sum_{cyc}(4a^6b+43a^5c^2+13a^4b^3+20a^5bc-103a^4b^2c+95a^4c^2b+108a^3b^3c-180a^3b^2c^2)\geq0.$$
Also, by Rearrangement: $$13\sum_{cyc}a^4b^3=13a^3b^3c^3\sum_{cyc}\frac{a}{c^3}\geq13a^3b^3c^3\sum_{cyc}\left(a\cdot\frac{1}{a^3}\right)=13\sum_{cyc}a^3b^3c,$$ $$95\sum_{cyc}a^4c^2b=95a^2b^2c^2\sum_{cyc}\frac{a^2}{b}\geq95a^2b^2c^2\sum_{cyc}a=95\sum_{cyc}a^3b^2c^2,$$ $$4\sum_{cyc}a^5c^2=4a^2b^2c^2\sum_{cyc}\frac{a^3}{b^2}\geq4a^2b^2c^2\sum_{cyc}a=4\sum_{cyc}a^3b^2c^2,$$ by Muirhead $$81\sum_{cyc}a^3b^3c\geq81\sum_{cyc}a^3b^2c^2$$ and by the Vasc's inequality $$(a^2+b^2+c^2)^2\geq3(a^3b+b^3c+c^3a),$$ which gives $$\sum_{cyc}(20a^5bc+40a^3b^3c)\geq60\sum_{cyc}a^4b^2c.$$ Id est, it's enough to prove that: $$\sum_{cyc}(4a^6b+39a^5c^2-43a^4b^2c)\geq0$$ and since $$\sum_{cyc}a^6b\geq\sum_{cyc}a^4b^2c,$$ we need to prove that: $$\sum_{cyc}(a^6b+13a^5c^2-14a^4b^2c)\geq0.$$ Now, let $a=\min\{a,b,c\}$, $b=a+u,$ $c=a+v$.
Thus, $$\sum_{cyc}(a^6b+13a^5c^2-14a^4b^2c)=60(u^2-uv+v^2)a^5+$$ $$+2(47u^3-51u^2v+60uv^2+47v^3)a^4+$$ $$+(66u^4-92u^3v+46u^2v^2+204uv^3+66v^4)a^3+$$ $$+(19u^5-13u^4v-56u^3v^2+130u^2v^3+116uv^6+19v^5)a^2+$$ $$+(u^6+6u^5v-14u^4v^2+65u^2v^4+26uv^5+v^6)a+u^2v(u^4+13v^4)\geq0.$$