The Contradiction method works!
Let $\frac{1}{\sqrt{a+1}}=p,$ $\frac{1}{\sqrt{b+1}}=q$ and $\frac{1}{\sqrt{c+1}}=r.$
Thus, $\{p,q,r\}\subset(0,1),$ $\frac{(1-p^2)(1-q^2)(1-r^2)}{p^2q^2r^2}=8$ and we need to prove that:
$$p+q+r<2.$$
Indeed, let $p+q+r\geq2,$ $r=kr'$ such that $k>0$ and $p+q+r'=2$.
Thus, $$p+q+kr'\geq2=p+q+r',$$ which gives $k\geq1.$
Thus, $$8=\frac{(1-p^2)(1-q^2)}{p^2q^2}\cdot\left(\frac{1}{k^2r'^2}-1\right)\leq\frac{(1-p^2)(1-q^2)}{p^2q^2}\cdot\left(\frac{1}{r'^2}-1\right),$$ which is a contradiction because we'll prove now that
$$8>\frac{(1-p^2)(1-q^2)}{p^2q^2}\cdot\left(\frac{1}{r'^2}-1\right).$$
Indeed, we need to prove that
$$8p^2q^2r'^2>(1-p^2)(1-q^2)(1-r'^2)$$ or
$$512p^2q^2r'^2>((p+q+r')^2-4p^2)((p+q+r')^2-4q^2)((p+q+r')^2-4r'^2)$$ or
$$512p^2q^2r'^2>(3p+q+r')(3q+p+r')(3r'+p+q)(p+q-r')(p+r'-q)(q+r'-p).$$
Now, if $(p+q-r')(p+r'-q)(q+r'-p)\leq0$, so our inequality is true, which says that it's enough to prove it for $(p+q-r')(p+r'-q)(q+r'-p)>0$.
Also, if $p+q-r'<0$ and $p+r'-q<0,$ so $p<0$, which is a contradiction.
Thus, we can assume that $p+q-r'=z>0,$ $p+r'-q=y>0$ and $q+r'-p=x>0$, which gives
$p=\frac{y+z}{2},$ $q=\frac{x+z}{2},$ $r'=\frac{x+y}{2}$ and we need to prove that
$$8(x+y)^2(x+z)^2(y+z)^2>xyz\prod_{cyc}(x+2y+2z),$$
which is obviously true after full expanding.
Done!
It's interesting that even the following is true.
Let $x$, $y$ and $z$ be non-negative numbers. Prove that:
$$125(x+y)^2(x+z)^2(y+z)^2\geq64xyz(x+2y+2z)(2x+y+2z)(2x+2y+z).$$
You don't require differentiability. The indefinite integral of any integrable decreasing function is concave, so $F=1-G$ is concave and $G$ is convex.
PS: There is link below given by Minus One-Twelfth which has a neat proof of convexity.
Best Answer
As per the comments, you missed an important part of the definition of concavity (and so Jensen's inequality): If $\lambda \in [0,1]$ then $$ f(\lambda x+(1-\lambda)y)\geq\lambda f(x)+(1-\lambda)f(y) $$ where you have taken $a=\lambda$, $b=1-\lambda$.
It may help to think of what this means geometrically: The points on the line between $f(x)$ and $f(y)$ (in the plane where $f$ is graphed), call one of them $(p_1,p_2)$ are smaller than the points on the graph, $(p_1,f(p_1))$.
I encourage you to draw the picture.