I think this is highly subjective, but I will try to answer based on my experience of solving some inequalities. Note that I am no expert by any means.
A lot of the times, using the correct function is kind of obvious. What is tricky, is figuring out the weights.
For example, as you say, the proof of AM-GM follows immediately from the convexity of $f(t) = \log(t)$. One way to see why this function works, is that the log function maps a multiplicative identity to an additive one, which is exactly what happens in the AM-GM inequality.
Another "trivial" inequality is
$$
\left(\frac{x_1+\cdots+x_n}{n} \right)^p\leq \frac{x_1^p+\cdots+x_n^p}{n},
$$
which again follows immediately from Jensen when applied to the function $t^p$, $p > 1$.
In other cases, the right function to use becomes obvious only after making some arrangements of the terms. This is especially the case if one can homogenize or normalize the inequality in some way. Take for example Minkowski's inequality:
$$
\left(\sum_{k=1}^n \lvert x_k + y_k\rvert^p\right)^{1/p}\leq \left(\sum_{k=1}^n \lvert x_k\rvert^p\right)^{1/p} + \left(\sum_{k=1}^n \lvert y_k\rvert^p\right)^{1/p}.
$$
It can in fact be derived if you choose $w_k=|y_k|^p/\sum_{j=1}^n|y_k|^p$, and $t_k = |x_k/y_k|^p$ in Jensen's inequality for the concave function $f(t) = (1+t^{1/p})^p$. One thing that gives this function away after rearranging is dividing both sides by $\left(\sum_{k=1}^n \lvert x_k\rvert^p\right)^{1/p}$.
Holder's inequality is another example of this, as pointed out by Gauge_name.
Another interesting way of using Jensen is in this answer: https://math.stackexchange.com/a/1033415/443030. Notice how homogenization works here.
Finally, one tricky inequality proven using Jensen is
$$
2\leq \frac{a_1}{a_2+a_3}+\frac{a_2}{a_3+a_4}+\frac{a_3}{a_4+a_1}+\frac{a_4}{a_1+a_2}.
$$
Divide both sides by $S=a_1+a_2+a_3+a_4$ and use the convexity of $f(t)=1/t$ to get
$$
\frac{1}{S}\left(\frac{a_1}{a_2+a_3}+\frac{a_2}{a_3+a_4}+\frac{a_3}{a_4+a_1}+\frac{a_4}{a_1+a_2}\right)\geq \frac{S}{a_1(a_2+a_3)+a_2(a_3+a_4)+a_3(a_4+a_1) + a_4(a_1+a_2)}.
$$
All that remains to prove is
$$
\frac{S^2}{a_1(a_2+a_3)+a_2(a_3+a_4)+a_3(a_4+a_1) + a_4(a_1+a_2)} \geq 2,
$$
which is easy.
In conclusion, as Calvin Lin pointed out in the comments, figuring out the correct way to use Jensen requires one to play around with the inequality until the convexity emerges. Certainly solving lots of inequalities goes a long way. However, two of the best tricks I found to work quite well is homogenization and normalization. These help a lot with figuring out the weights and values to be used in Jensen, and to a large extent helps to find a "tight" function to use.
Best Answer
I don't think you can prove this using strict convexity. If you use the following addition al fact about the process $(Y_t)$ you can prove this from first principles:
$$0<P(Y_t <K) <1$$
Assuming this note that $\int Y_t dP=\int_{Y_t \geq K} Y_t dP+\int_{Y_t < K} Y_t dP<\int_{Y_t \geq K} Y_t dP+KP(Y_t<K)$: strict inequality holds because $\int_{Y_t < K} Y_t dP \leq KP(Y_t <k) <K$ by above mentioned property.
Keeping this in mind consider $Eg(Y_t)=\int (Y_t-K)^{+}dP =\int_{Y_t \geq K} (Y_t-K) dP>\int Y_t dP-K$ if $EY_t >K$ (by the inequality just established); also $\int_{Y_t \geq K} (Y_t-K) dP>0$ (by the fact that $P(Y_t \geq K) >0$. Combining these two inequalities we get $Eg(Y_t) > \max \{0, \int Y_t dP-K\}=g(EY_t)$.