[Math] Convex conjugate of perspective

convex optimizationconvex-analysis

I am trying to solve this problem:

Express the conjugate of the perspective of a convex function f in terms of $f^*$

Defintion 1:

The (convex) conjugate of a function is given by:
$$f^*(y)=\sup_{x\in dom ~f}(y^Tx-f(x))$$

Definition 2:

The perspective of a function $f:\mathbb{R}^n\to \mathbb{R}$ is the function $g:\mathbb{R}^n \times \mathbb{R} \to \mathbb{R}$ where $g(x,t)=tf(\dfrac{x}{t})$ and $dom~g = \{(x,t)| \dfrac{x}{t} \in dom ~f and t>0\}$

My attempt:

First, we express $g(x,t)$ as $g(x_t)$ where $x_t$ is an augmented vector variable $x_t = [x,t] = [x_1, …, x_n, t]$.

The conjugate of $g(x_t)$ is given as:
$$g^*(y,s)=\sup_{\frac{x}{t}\in dom~f\\ t>0}(t(\dfrac{y^Tx}{t}+s-f(\dfrac{x}{t})))$$

I don't know how to take care of supremum when it has two variables $x/t$ and $t$

Best Answer

$$\sup_{\frac{x}{t}\in dom~f\\ t>0}(t(\dfrac{y^Tx}{t}+s-f(\dfrac{x}{t}))) = \sup\{tv \mid v =\sup_{u\in dom~f}((y^Tu+s-f(u)))\text{ and }\color{red}{t > 0}\}$$

Edit to address a question in the comments

What does $$\sup_{\frac{x}{t}\in dom~f\\ t>0}(t(\dfrac{y^Tx}{t}+s-f(\dfrac{x}{t})))$$ mean?

It means we form a set $$A =\left\{t(\dfrac{y^Tx}{t}+s-f(\dfrac{x}{t}))~ \right|~\left. t > 0, \frac xt \in dom~f\right\}$$ Then we find its supremum.

But that is a bit ungainly. After all, everywhere that $x$ occurs in the definition of $A$, it is as a part of $x/t$. So why not just use that value directly instead of expressing it in terms of this other arbitrary value $x$? So I replaced $x/t$ with $u$ to get $$A = \left\{t(y^Tu+s-f(u))\mid t > 0, u \in dom~f\right\}$$

For the next step, I just made use of this equivalence:

$$\sup\{\phi(r,s)\mid r\in A, s\in B\} = \sup\{\sup\{\phi(r,s)\mid s\in B\}\mid r \in A\}$$

where $\phi : A\times B \to \Bbb R$. This is easy enough to see: For any value $p$ in the LH set, $p = \phi(r,s)$ for some $r,s$, and so $p\le \sup\{\phi(r,s)\mid s\in B\} \le \sup\{\sup\{\phi(r,s)\mid s\in B\}\mid r \in A\}$. Thus the LH side is $\le$ the RH side. Conversely, For any value $h$ less than the RH side, there must be an $r \in A$ with $\sup\{\phi(r,s)\mid s\in B\} > h$, which in turn means there is an $s \in B$ with $\phi(r,s) > h$, which means that the LH side is also $> h$. Hence they must be equal.

Of course this leaves you with a problem, which wasn't apparent in my original answer, because I wrote down the wrong thing. You will note that originally I had $tv \in dom~f$. I had meant this to be $t > 0$ (which correction is now highlighted in red, if your browser shows colored mathjax) but in my various editings of the form, I had accidently left in the wrong thing and deleted the correct one.

But in this form, it is quite clear. Since $t$ can be arbirarily large, the common value of these two expressions is $\infty$ (assuming that $f^*$ has at least one positive value). Once it is made obvious by my version, you can check that it holds in your version as well. Pick an arbitrary $u \in dom~f$ with $f(u) > 0$ and for any $t > 0$, you can take $x = tu$ to make the supremum as large as you like.

There are two possibilities here. Either the point of this exercise is for you to discover the complex conjugate of a perspective function is always infinite, or else, by conjugate of the perspective function, they only mean to conjugate with respect to constant $t$:

$$ g^*(y,t)=\sup_{\frac{x}{t}\in dom~f}(t(\dfrac{y^Tx}{t}+s-f(\dfrac{x}{t})))$$

I can't say which is the case for your course.