Problem statement
Use the definition of convexity of a function, i.e., that for any $\boldsymbol{x}$, $\boldsymbol{y} \in \mathbb{R}^{d}$ and $\lambda \in \left [0,1 \right ]$ we have
\begin{align*}
f(\lambda \boldsymbol{x} +(1-\lambda)\boldsymbol{y} ) \leq \lambda f(\boldsymbol{x}) + (1-\lambda)f(\boldsymbol{y})
\end{align*}
to show that if f is convex and differentiable at $\boldsymbol{x}$ then
\begin{align*}
f(\boldsymbol{y}) \geq f(\boldsymbol{x}) + \nabla f(\boldsymbol{x})^{\top} (\boldsymbol{y}-\boldsymbol{x})
\end{align*}
for all $\boldsymbol{y} \in \mathbb{R}^{d}$ (Use the definition of the directional derivative)
In order to get to the desired result, I have tried using the definition of a convex function together with an illustration. I am unsure whether my reasoning is correct and believe that there must be a way to derive this mathematically, but unfortunately I don't really have a strong maths background. I have found a similar question here, but it doesn't really answer my question.
Attempted proof
Summary
I have tried to prove this by illustration, but am looking for an analytical solution.
Any help will be greatly appreciated 🙂
Best Answer
We have \begin{align*} f(\lambda \boldsymbol{x} +(1-\lambda)\boldsymbol{y} ) \leq \lambda f(\boldsymbol{x}) + (1-\lambda)f(\boldsymbol{y}) \end{align*}
For scalar case divide above by $1-\lambda$
$f(y)=f(x)+\frac{f(x+(1-\lambda)(y-x))-f(x)}{1-\lambda}$.
Then take limit as $1-\lambda\rightarrow0$ to get what you want.