This really is a case of grinding through.
(i) $f^*(\alpha) = \sup_x \alpha x - e^x$. Note that for $\alpha <0$, if $x<0$ we have $\alpha x -e^x > \alpha x -1$ and so $f^*(\alpha) = \infty$.
If $\alpha = 0$, then choose large negative $x$ we see that $f^*(0) = 0$. Finally, as you have done in the question, for $\alpha>0$ we have
$f^*(\alpha) = \alpha( \log \alpha -1)$.
Hence $f^*(\alpha) = \begin{cases}\infty, & \alpha < 0 \\0, & \alpha =0 \\
\alpha( \log \alpha -1), & \alpha >0 \end {cases}$.
(ii) $f^*(\alpha) = \sup_x \sum_k \alpha_k x_k - \rho |x_k|$.
It is straightforward to see that if
$\| \alpha\|_\infty > \rho$ then $f^*(\alpha) =\infty$.
If $\| \alpha\|_\infty \le \rho$, then
$f^*(\alpha) \le \sup_x \sum_k (|\alpha_k| - \rho )|x_k| \le 0$, so $f^*(\alpha) = 0$.
Hence $f^*(\alpha) = \begin{cases} 0,& \|\alpha\|_\infty \le \rho \\
\infty, & \|\alpha\|_\infty > \rho\end {cases}$.
In view of the question you linked in your question above (and answers thereof), we know that if $f$ is convex l.s.c, then
$f(x) + f^*(u) = \langle x,u\rangle \iff u \in \partial f(x)$, where $\partial f(x) \subseteq X^*$ is the subdifferential of $f$ at $x$ defined by
$$
\partial f(x) := \{x^* \in X^* \mid f(x') \ge f(x) + \langle x^*, x'-x\rangle\;\forall x' \in X\}
$$
Here $X^*$ is the topological dual of the real (or complex!) Banach space $X$, the space of all linear functionals $X \mapsto \mathbb R$.
The mapping $\langle\cdot,\cdot\rangle : X^* \times X \rightarrow \mathbb R$ is the duality bracket. We consider Banach spaces and not just Hilbert spaces because we want to be as general as reasonable (think about function spaces, spaces of probability distributions, etc.).
We'll assume $X$ is reflexive, which means $X^{**} = X$. A crash course on convex analysis in Banach spaces can be found here.
To answer the first part of your question in the affirmative, it then suffices to prove that
Lemma. If $f$ is convex and differentiable at $x$, then $\partial f(x) = \{\nabla f(x)\}$, i.e $\nabla f(x)$ is the unique subgradient of $f$ at $x$.
Proof. See https://math.stackexchange.com/a/1989140/168758. $\quad\quad\Box$
This proves that
The choice $g=f^*$ solves the functional equation
$$
\begin{split}
&f(x) + g^\star(\nabla f(x)) = \langle x, \nabla f(x)\rangle\;\forall x \in \text{intdom}\;f,\\
&g:\text{intdom}\;f^* \rightarrow X,
\end{split}
\tag{FY}
$$
where $\text{intdom}\;f$ is the interior of $\text{dom}\;f := \{x \in X \mid f(x) < +\infty\}$.
On the uniqueness of $f^*$ as a solution of (FY)
Suppose that $f$ is a Legendre function, meaning that
- $f$ is convex l.s.c and continuously differentiable on $\text{intdom}\; f$, and
- the gradient-mapping $\nabla f: \text{intdom}\;f \rightarrow X^*$ is one-to-one (a sufficient condition is that this $f$ is essentially strictly convex; see Fact 2.4 and Corollary 2.6 of https://people.ok.ubc.ca/bauschke/Research/07.pdf).
Then $f^*$ is also a Legendre function and one has the following conjugate-inversion formulae (see Rockafellar's Convex Analysis, Theorem 26.5 for proof!)
$\nabla f^*\nabla f=\text{ the identity operator on }\text{int}\text{dom}\;f$,
$\nabla f\nabla f^* = \text{ the identity operator on }\text{int}\text{dom}\;f^*$.
Now, back to our problem. Suppose $g:\text{int}\;\text{dom}\;f^* \rightarrow X$ is another function which satisfies (FY), and let $u \in \text{int}\;\text{dom}\;f^*$. Then by the above conjugate-inversion formula, we have $u=\nabla f(x_u)$ where $x_u = \nabla f^*(u) \in \text{intdom}\;f$. Thus
$$
g(u) = g(\nabla f (x_u)) = \langle x_u ,\nabla f(x_u)\rangle - f(x_u) = f^*(\nabla f(x_u)) = f^*(u),
$$
where the 2nd and 3rd equalities are due to the fact that $g$ and $f^*$ solve (FY).
The above equation shows that $g=f^*$ on $\text{intdom}f^*$, thus proving the unicity of $f^*$ as solution to the functional equation (FY). $\quad\quad\Box$
We have the hammer, where are the nails ?
Note that a by-product of our above analysis is that
The conjugate $f^*$ can be explicity computed like so
$$
f^*(u) = \langle (\nabla f)^{-1}(u),u\rangle - f((\nabla f)^{-1}(u)),
\;\forall u \in \text{intdom} f^*.
\tag{1}
$$
As an example, let's prove the popular statement
"Log-sum-exp" is the convex conjugate of "relative entropy".
More precisely, let $X$ be the real Banach space of Radon measures on a topological space $\Omega$ (e.g a finite set), so that $X^*$ is the space of continuous functions on $\Omega$ equipped with the sup-norm, and the duality bracket is given for every $(v,\nu) \in X^* \times X$ by $\langle \nu, v\rangle = \mathbb E_\nu[v] := \int_\Omega vd\nu$. Now, fix a base measure $\mu$ on $\Omega$, and consider the so-called relative entropy functional (relative to the base measure $\mu$)
$$
\begin{split}
H(\cdot\|\mu): X &\rightarrow (-\infty,+\infty],\\
\nu &\mapsto H(\nu \| \mu) := \begin{cases}\mathbb E_{\nu}\left[\log\left(\frac{d\nu}{d\mu}\right)\right] = \int_{\Omega}\log\left(\frac{d\nu}{d\mu}\right)d\nu,&\mbox{ if }\nu \ll \mu,\\+\infty,&\mbox{ else.}\end{cases}
\end{split}
$$
Here, $\dfrac{d\nu}{d\mu}$ is the Radon-Nikodym derivative of $\nu$ w.r.t $\mu$.
One can easily check that this is a Legendre function. Now, let $\nu$ be a probability distribution on $\Omega$ with $\nu \ll \mu$, and let $u:\Omega \rightarrow \mathbb R$ be a continuous function. The following computations are trivial
- $\nabla f(\nu) = \log\left(\frac{d\nu}{d\mu}\right) + 1$ (pointwise logarithm), and so $\nabla f(\nu) = u$ iff $d\nu = (e^{u}/Z)d\mu$, where $Z_u = \mathbb E_\mu[e^u]$.
- Thus $(\nabla H(\cdot\|\mu))^{-1}(u) \ll \mu$ with Radon-Nikodym derivative equal to $e^u/Z_u$.
Plugging into formula (1) with $f=H(\cdot\|\mu)$, we get
$$
\begin{split}
H(\cdot\|\mu)^*(u) &= \langle (\nabla H(\cdot\|\mu))^{-1}(u),u\rangle - f((\nabla H(\cdot\|\mu))^{-1}(u))\\
&= \int_\Omega (ue^u/Z_u) d\mu - \int_\Omega \left(e^u/Z_u\right)\log\left(e^u/Z_u\right)d\mu\\
&= \mathbb E_{\mu}[ue^u/Z_u] - \mathbb E_{\mu}[ue^u/Z_u] + \log(Z_u)
= \log(Z_u) = \log(\mathbb E_\mu[e^u]).
\end{split}
$$
How cool is that ?
Best Answer
By definition, we have $f^*(z) = \underset{x \in \mathbb{R}}{\sup} \: zx - f(x)$. Since $f(x) = +\infty$ for $\left\lvert x \right\rvert > a$, we can restrict the sup to $\left\lvert x \right\rvert \leq a \implies f^*(z) = \underset{\left\lvert x \right\rvert \leq a}{\sup} \: zx + \sqrt{a^2 - x^2}$. Letting $y = \frac{x}{a}$, we get $f^*(z) = a \left(\underset{\left\lvert y \right\rvert \leq 1}{\sup} \: zy + \sqrt{1 - y^2}\right)$. Substituting $y = \sin(\theta)$, we obtain $f^*(z) = a \left(\underset{\theta}{\sup} \: z\sin(\theta) + \cos(\theta)\right)$. Noting that $z\sin(\theta) + \cos(\theta) = \sqrt{1+z^2} \cos(\phi)$ for some appropriately defined $\phi$, we deduce that $f^*(z) = a\sqrt{1+z^2}$.