A differentiable function $f: \mathbb{R}^n \rightarrow \mathbb{R}$ is said to have Lipchitz continuous gradient or is L-LG if the following holds for some $L>0$:
$$
\|\nabla f(\mathbf{x}) – \nabla f(\mathbf{y})\|\leq L\|\mathbf{x}-\mathbf{y}\|,\, \forall \mathbf{x},\mathbf{y} \in \mathbb{R}^n \label{1}\tag{1}.
$$
Now, consider a sigmoid function $\sigma : \mathbb{R}\to [0,1]$ where $\sigma(x)=\frac{e^x}{1+e^x}$ whose derivative is the following:
$$
\nabla_x \sigma(x)=\sigma'(x)= \sigma(x) -\sigma^2(x)
$$
where $\sigma^2(x)=(\sigma(x))^2$.
Is sigmoid function L-LG? In other words, is there any $L>0$ such that the following holds for all $x, y \in \mathbb{R}^n$?
$$
|\sigma'(y) – \sigma'(x)|\leq L |y-x|
$$
Best Answer
Yes, $\sigma'$ is Lipschitz because any function with bounded derivative is Lipschitz, and $\sigma''$ is bounded. It looks like you're defining "$\sigma$ is L-LG" to mean the same as "$\sigma'$ is Lipschitz".
Edit: There's a relevant MSE answer here.