Jensen's inequality states that for a convex function $f:I \to \mathbb R$ and $x_1,…,x_n\in I$, we have $$\frac{1}{n}\sum_{i=1}^n f(x_i)\ge f\left(\frac{1}{n}\sum_{i=1}^n x_i\right)$$
But in order to use this inequality, you have to choose the right function. Take for example this problem
$x_1,…,x_n$ are positive reals such that $$\sum_{i=1}^n\frac{1}{1+x_i}=1$$ Show that $$\prod_{i=1}^nx_i\ge (n-1)^n$$
Here the function that we're looking for turns out to be $f(x)=\frac{1}{1+e^x}$
In the proof of the AM-GM inequality, we've used $f(x)=\ln x$.
So is there a way to know what's the function that will help me solve my inequality?
Best Answer
I think this is highly subjective, but I will try to answer based on my experience of solving some inequalities. Note that I am no expert by any means.
A lot of the times, using the correct function is kind of obvious. What is tricky, is figuring out the weights.
For example, as you say, the proof of AM-GM follows immediately from the convexity of $f(t) = \log(t)$. One way to see why this function works, is that the log function maps a multiplicative identity to an additive one, which is exactly what happens in the AM-GM inequality.
Another "trivial" inequality is $$ \left(\frac{x_1+\cdots+x_n}{n} \right)^p\leq \frac{x_1^p+\cdots+x_n^p}{n}, $$ which again follows immediately from Jensen when applied to the function $t^p$, $p > 1$.
In other cases, the right function to use becomes obvious only after making some arrangements of the terms. This is especially the case if one can homogenize or normalize the inequality in some way. Take for example Minkowski's inequality: $$ \left(\sum_{k=1}^n \lvert x_k + y_k\rvert^p\right)^{1/p}\leq \left(\sum_{k=1}^n \lvert x_k\rvert^p\right)^{1/p} + \left(\sum_{k=1}^n \lvert y_k\rvert^p\right)^{1/p}. $$ It can in fact be derived if you choose $w_k=|y_k|^p/\sum_{j=1}^n|y_k|^p$, and $t_k = |x_k/y_k|^p$ in Jensen's inequality for the concave function $f(t) = (1+t^{1/p})^p$. One thing that gives this function away after rearranging is dividing both sides by $\left(\sum_{k=1}^n \lvert x_k\rvert^p\right)^{1/p}$.
Holder's inequality is another example of this, as pointed out by Gauge_name.
Another interesting way of using Jensen is in this answer: https://math.stackexchange.com/a/1033415/443030. Notice how homogenization works here.
Finally, one tricky inequality proven using Jensen is $$ 2\leq \frac{a_1}{a_2+a_3}+\frac{a_2}{a_3+a_4}+\frac{a_3}{a_4+a_1}+\frac{a_4}{a_1+a_2}. $$ Divide both sides by $S=a_1+a_2+a_3+a_4$ and use the convexity of $f(t)=1/t$ to get $$ \frac{1}{S}\left(\frac{a_1}{a_2+a_3}+\frac{a_2}{a_3+a_4}+\frac{a_3}{a_4+a_1}+\frac{a_4}{a_1+a_2}\right)\geq \frac{S}{a_1(a_2+a_3)+a_2(a_3+a_4)+a_3(a_4+a_1) + a_4(a_1+a_2)}. $$ All that remains to prove is $$ \frac{S^2}{a_1(a_2+a_3)+a_2(a_3+a_4)+a_3(a_4+a_1) + a_4(a_1+a_2)} \geq 2, $$ which is easy.
In conclusion, as Calvin Lin pointed out in the comments, figuring out the correct way to use Jensen requires one to play around with the inequality until the convexity emerges. Certainly solving lots of inequalities goes a long way. However, two of the best tricks I found to work quite well is homogenization and normalization. These help a lot with figuring out the weights and values to be used in Jensen, and to a large extent helps to find a "tight" function to use.