Let $A$ be an open convex subset of a normed space. If $f:A \to \mathbb R$ is convex and continuous at $a$, then $f’_+(a, \cdot)$ is Lipschitz

convex-analysisderivativesfunctional-analysisnormed-spaces

This thread is meant to record a question that I feel interesting during my self-study. I'm very happy to receive your suggestion and comments.

See: SE blog: Answer own Question and MSE meta: Answer own Question. Anyway, it is written as problem.

Let $X$ be a normed space, $A \subset X$ an open set, $f: A \to \mathbb{R}$ a function, and $a \in A$ a point. For a "direction" $v \in X$, we shall consider the right directional derivative $f_{+}^{\prime}(a, v)$, the left directional derivative $f_{-}^{\prime}(a, v)$, and the (bilateral) directional derivative $f^{\prime}(a, v)$, which are defined by:
$$
\begin{aligned}
f_{+}^{\prime}(a, v) &=\lim _{t \to 0+} \frac{f(a+t v)-f(a)}{t} \\
f_{-}^{\prime}(a, v) &=\lim _{t \to 0-} \frac{f(a+t v)-f(a)}{t} \\
f^{\prime}(a, v) &=\lim _{t \to 0} \frac{f(a+t v)-f(a)}{t}.
\end{aligned}
$$

We shall say that $f$ is:

  • Gâteaux differentiable at $a$ if $f^{\prime}(a, \cdot) \in X^*$ (that is, $f^{\prime}(a, \cdot)$ is everywhere defined, real-valued, linear and continuous);
  • Fréchet differentiable at $a$ if there exists $x^{*} \in X^{*}$ such that
    $$
    \lim _{h \to 0} \frac{f(a+h)-f(a)-x^{*}(h)}{\|h\|}=0 .
    $$

Theorem: Let $X$ be a normed space, $A \subset X$ an open set, $f: A \to \mathbb{R}$ convex. Fix $a\in A$. Then

  • (a) $f_{+}^{\prime}(a, v)$ exists finite for each $v \in X$, and the functional $p=f_{+}^{\prime}(a, \cdot)$ is sublinear on $X$.
  • (b) $-p(-v) \leq p(v)$ for each $v \in X$.
  • (c) The set
    $$
    V=\left\{v \in X \mid f^{\prime}(a, v) \text { exists }\right\}=\{v \in X \mid -p(-v)=p(v)\}
    $$
    is linear (that is, $V$ is a subspace of $X$ ) and the restriction $\left.p\right|_{V}$ is linear.
  • (d) If $f$ is continuous at $a$, then $p$ is Lipschitz (in particular, $\left.p\right|_{V} \in V^{*}$ ).

A direct corollary is as follows.

Let $X$ be a normed space, $A \subset X$ an open set, $f: A \to \mathbb{R}$ convex and continuous at $a\in A$. If $f^{\prime}(a, v)$ exists for all $v \in X$, then $f$ is Gâteaux differentiable at $a$.

Best Answer

  • (a) Fix $v \in X$. There is $\varepsilon>0$ such that $a+tv \subset A$ for all $t$ such that $|t| < \varepsilon$. Consider the map $$ \varphi: (-\varepsilon, \varepsilon) \to \mathbb R, t \mapsto f(a+tv). $$

Then $\varphi$ is convex. Let $-t_1<0<t_1<t_2<\varepsilon$. By the chordal slope lemma, we get $$ \frac{\varphi(0) - \varphi(-t_1)}{t_1} \le\frac{\varphi(t_1) - \varphi(0)}{t_1} \le \frac{\varphi(t_2) - \varphi(0)}{t_2} \le \frac{\varphi(t_2) - \varphi(t_1)}{t_2-t_1}. $$

It follows that the map $$ (0, \varepsilon) \to \mathbb R, t \mapsto \frac{f(a+tv)-f(a)}{t} $$ is non-decreasing and bounded from below. Then $f'_+(a, v)$ exists for all $v\in X$. We have $$ \frac{f(a+t(\lambda v))-f(a)}{t} = \lambda \frac{f(a+(\lambda t)v)-f(a)}{\lambda t} \quad \forall \lambda >0, $$ so $p$ is positively homogeneous. Let $v_1, v_2 \in X$. We have $$ f(a+t(v_1+v_2)) = f \left ( \frac{(a+2tv_1) + (a+2tv_2)}{2} \right ) \le \frac{1}{2} f(a+2tv_1) + \frac{1}{2} f(a+2tv_2) $$ Then $$ \frac{f(a+t(v_1+v_2))-f(a)}{t} \le \frac{f(a+2tv_1)-f(a)}{2t} + \frac{f(a+2tv_2)-f(a)}{2t}. $$

We take the limit $t \to 0^+$ and get that $p$ is sub-additive.

  • (b) As shown above, $$ \frac{f(a)-f(a-t_1v)}{t_1} = \frac{\varphi(0) - \varphi(-t_1)}{t_1} \le\frac{\varphi(t_1) - \varphi(0)}{t_1} = \frac{f(a+t_1v) - f(a)}{t_1}. $$ Then $$ -\frac{f(a+t_1(-v)) - f(a)}{t_1} \le \frac{f(a+t_1v) - f(a)}{t_1}. $$ The claim then follows by taking the limit $t_1 \to 0^+$.

  • (c) Notice that $f'_-(a, v) = -f'_+(a, -v) = -p(-v)$ and that $v\in V \iff -v\in V$.

Let $\lambda \in \mathbb R$ and $v \in V$. If $\lambda \ge 0$, then $p(-\lambda v) = \lambda p(-v) = -\lambda p(v) = -p(\lambda v)$ because $v\in V$. Now consider $\lambda<0$. We have $p(-\lambda v) = p(-(-\lambda) (-v)) = - p((-\lambda)(-v)) = - p(\lambda v)$ because $-\lambda>0$ and $-v\in V$. So $\lambda v \in V$.

Let $v_1, v_2 \in V$. By (b), we have $p(v_1+v_2) \ge -p(-v_1-v_2)$. Because $p$ is sub-additive, $p(v_1+v_2) \le p(v_1)+p(v_2) = -p(-v_1)-p(-v_2) = -[p(-v_1)+p(-v_2)] \le -p(-v_1-v_2)$. Hence $p(v_1+v_2) = -p(-v_1-v_2)$. So $v_1+v_2 \in V$. It follows that $V$ is a linear subspace.

Let $\lambda \in \mathbb R$ and $v \in V$. If $\lambda \ge 0$, then $p(\lambda v) =\lambda p(v)$. Now consider $\lambda<0$. We have $p(\lambda v) =p((-\lambda)(-v)) = -\lambda p(-v) =-\lambda(-p(v)) = \lambda p(v)$ because $-\lambda>0$ and $-v\in V$.

Let $v_1, v_2 \in V$. We have $p(v_1+v_2) \le p(v_1)+p(v_2)$ by sub-additivity. On the other hand, $p(v_1+v_2)=p(-(-v_1-v_2))=-p(-v_1-v_2) \ge -[p(-v_1)+p(-v_2)] = -[-p(v_1)-p(v_2)] = p(v_1)+p(v_2)$. Hence $p(v_1+v_2)=p(v_1)+p(v_2)$. Hence $p$ is linear on $V$.

  • (d) We need the following Lemma.

Lemma Let $(X, \| \cdot\|)$ be a normed vector space, $C$ its open convex subset, and $f:C \to \mathbb R$ convex. Then the following statements are equivalent.

  • (i) $f$ is locally Lipschitz on $C$;
  • (ii) $f$ is continuous on $C$;
  • (iii) $f$ is continuous at some point of $C$;
  • (iv) $f$ is locally bounded on $C$;
  • (v) $f$ is upper bounded on a nonempty open subset of $C$.

It follows that $f$ is $L$-Lipschitz on $A$ for some $L>0$. We have $$ \begin{align} |p(v_1)-p(v_2)| &= \left | \lim_{t\to 0^+} \frac{f(a+tv_1) - f(a+tv_2)}{t} \right | \\ &= \lim_{t\to 0^+} \frac{|f(a+tv_1) - f(a+tv_2)|}{t} \\ &\le \lim_{t\to 0^+} \frac{Lt\|v_1-v_2\|}{t} \\ &= L\|v_1-v_2\|. \end{align} $$

This completes the proof.

Related Question