Let $f:\mathbb{R}^n\rightarrow\overline{\mathbb{R}}$ be a proper closed convex function that is locally Lipschitz continuous on its domain $D(f)$. Define the proximal mapping of $f$ to be
$$\textbf{prox}_{\lambda f}(x)=\arg\min_u\left\{f(u)+\frac{1}{2\lambda}\|u-x\|^2\right\}$$
In this context, it is not hard to see $\textbf{prox}_{\lambda f}(x)$ is a Lipschitz continuous function w.r.t. variable $x$ for any parameter $\lambda>0$.
I am wondering under what assumption on $f$, will the directional derivative of proximal mapping, i.e.,
$$\textbf{prox}_{\lambda f}'(x;d) := \lim_{t\downarrow 0}\frac{\textbf{prox}_{\lambda f}(x+td)-\textbf{prox}_{\lambda f}(x)}{t}$$
exist?
From the existing literature (Proposition 5.3.5, p. 141), I know if $f:=I_C$ is an indicator function of a closed convex set $C$ (hence proximal mapping reduces to projection mapping), then the directional derivative exists. To be more precise,
for any $x\in C$ and $d\in\mathbb{R}^n$,
$$\textbf{proj}_C'(x;d) := \lim_{t\downarrow 0}\frac{\textbf{proj}_C(x+td)-\textbf{proj}_C(x)}{t} = \textbf{proj}_{T_C(x)}(d)$$
where $T_C(x)$ is the tangent cone of $C$ at $x$.
I am wondering if this is only special for projection operator or can be slightly generalized to proximal operator of convex functions with some nice properties or structures.
Best Answer
You must be careful, in your reference, the directional differentiability of the projection onto $C$ is only established for points $x \in C$. In fact, the projection might fail to be directionally differentiable at points outside $C$. For a counterexample, see the paper "Directionally nondifferentiable metric projection" by A. Shapiro.
For a general function $f$, one can prove that the directional differentiability of the prox is equivalent to $f$ being twice epi-differentiable. This should be in the book "Variational Analysis" by Rockafellar and Wets.
Indeed, one can argue as in Corollary 13.43 by using Theorem 13.40. Let $f$ be convex and twice epidifferentiable. Fix $u$, set $\newcommand\prox{\operatorname{prox}}x = \prox_f(u)$ and $v = u - x$. We have $$ \begin{split} \newcommand\dd{\mathrm{d}} \prox_f &= (I + \partial f)^{-1}\\ D(\partial f)(x,v) &= \partial [\frac12\dd^2f(x,v)]\\ D\prox_f(u,x) &= D(I + \partial f)^{-1}(u,x) = (D(I + \partial f)(x,u))^{-1} \\&= [DI(x,x) + D\partial f(x,u-x)]^{-1} \\ &=\partial [I + \partial\frac12\dd^2f(x,v)]^{-1} \\&= \prox_{\frac12 \dd^2 f(x,v)}. \end{split} $$ Hence, $z = \prox_f'(u; w) = D\prox_f(u,x)$ if $z$ minimizes $$ \frac12 \|z - w\|^2 + \frac12 \dd^2 f(x,v)(z). $$