Let $f: \mathbb{R}_+ \rightarrow \mathbb{R}$ where $f(x)=x\log(x)$. How to show it is strongly convex, i.e.,
Definition: Let $f: \mathbb{R}^n \rightarrow \mathbb{R}$ be differentiable. Then $f$ is strongly convex if $\exists$ a positive constant $\alpha > 0$ such that
$$
\langle \nabla f(y) – \nabla f(x),y-x \rangle \geq \alpha||y-x||^2 \,\,\,\,\,,\,\,\,\forall x,y \in \mathbb{R}^n \tag{1}
$$
Following $(1)$ we have $(y-x)\log(\frac{y}{x})\geq \alpha(y-x)^2
\,\,\,\,\,
\forall x,y \in \mathbb{R}_+$.
What $\alpha$ satisfies the above and how we can handle the inequality $\forall x,y \in \mathbb{R}_+$ ?
Best Answer
We have $f'(x)=1+\log(x)$. Strict convexity therefore means that there exists a strictly positive $\alpha$ such that $$[\log(y)-\log(x)](y-x)\geq\alpha(y-x)^2$$ holds. Without loss of generality I assume $y>x$. Then the above inequality requires that $$\log(y)-\log(x)\geq\alpha(y-x).$$ Although you don't state it, I assume that the variables $x$ and $y$ live on $(0,1)$ (because they are probabilities). Since the slope of the $\log$ function on the interval $(0,1)$ is larger than or equal to 1, you can choose any positive $\alpha$ smaller than 1.
If $x$ and $y$ live on a general bounded interval $(0,M)$, then the argument goes through with $\alpha<1/M$.