[Math] First order condition for a convex function

convex optimizationconvex-analysis

The first order condition for convexity of a function states that:
For a convex function $f$ over a convex domain:
$$
f(y)\geq f(x) + \nabla f(x)^T(y-x).
$$
Actually this is an iff, but lets leave the other side here.
There are plenty of proofs for this online, with an example appearing here for reference.
All proofs I've seen do the same:

  1. Use the definition of convexity
  2. Divide both sides by $t$
  3. Assume $t$ is infinitesimally small
  4. Replace the fraction by the gradient

My question is as follows:
What is the source of the legitimacy for assuming a small $t$? The statement should hold for $t\in[0,1]$?

Best Answer

We have the relation $$\frac{f(x+\lambda(y-x))-f(x)}{\lambda}\leq f(y)-f(x)$$ Then as $\lambda$ grows, the LHS gets smaller. Therefore we are interested in the what happens for the smallest $\lambda$ possible, and that is the reason to take $\lambda \to 0^+$. Everything that holds as $\lambda$ tends to $0$ also holds for larger values.

Related Question