Solved – Relationship between first and second order condition of convexity


Suppose we have a function $f(\boldsymbol{x})$ and its hessian, i.e $\nabla_{\boldsymbol{x}}^2f(\boldsymbol{x})$, equals $\mathbf{0}$.

We know that for convexity $\nabla_{\boldsymbol{x}}^2f(\boldsymbol{x})\succeq\mathbf{0}$.

The first question is whether $f(\boldsymbol{x})$ is strictly convex or just convex? Secondly, if the hessian is equal to $\mathbf{0}$, how does it change the first order condition of convexity? Does the inequality in the first order condition $f(\boldsymbol{y})\geq f(\boldsymbol{x}) + \nabla^\top f(\boldsymbol{x}) (\boldsymbol{y}-\boldsymbol{z})$ also reduces to equality?

Maybe if someone can tell how the second order condition is derived from the first order condition, it could answer my question.

Best Answer

A real valued function is convex, using the first-order condition, if the inequality below holds

$$ f(\alpha x +(1-\alpha)y) \leq \alpha f(x) +(1-\alpha) f(y) $$

It is strictly convex if such inequality holds for $<$.

Now, the second-order condition can only be used for twice-differentiable functions (after all you'll need to be able to compute it's second derivatives), and strict convexity is evaluted like above; convex if $\nabla_{\boldsymbol{x}}^2f(\boldsymbol{x})\succeq\mathbf{0}$ and strictly convex if $\nabla_{\boldsymbol{x}}^2f(\boldsymbol{x})>\mathbf{0}$.

Finally, the second-order condition does not overlap the first-order one, as in the case of linear functions.

Related Question