Non-convex functions and their supporting hyperplanes

convex-analysisconvex-geometrygeometry

Consider a continuous function $L(x)$ over a convex compact subset $D$ of $\mathbb{R}^n$. Its epigraph is the set of points $(x,y)$ where $y\geq L(x)$. If $L$ is convex then every point in $D$ lies under a point on the graph of $L$ through which a supporting hyperplane to the epigraph passes. But for non-convex functions there are some points on the graph through which no supporting hyperplane passes (they are on the "dents" of the graph). Those are the points where the graph of the second convex conjugate $L^{**}$ detaches from the graph of $L$. Let us call points where the graphs coincide support points.

Is every point in the interior of $D$ a convex combination of the $x$-projections of support points on the same hyperplane that supports the epigraph of $L^{**}$ at it? It is trivially true for convex functions, in 1D any point is always a convex combination of at most two such projections, and it seems intuitive in general (perhaps it is at most $n+1$ points for $\mathbb{R}^n$). This might be related to the Minkowski-Carathéodory theorem, but I am not sure how. Is there a standard result that implies this?

Any point on the graph of $L^{**}$ has a supporting hyperplane to the graph of $L$ passing through it, so this hyperplane must pass through some points on the graph of $L$. They are support points. But why is the $x$-projection of the original point a convex combination of their $x$-projections? In 1D this is because it has to lie between them unless it is one of them.

Best Answer

Suppose $x_0$ is a strongly-exposed point of $D$. That is, there exists some linear functional $f \in (\Bbb{R}^n)^*$ such that $f \mid_D$ attains its maximum at $x_0$, and given any $(x_m) \in D$ such that $f(x_m) \to f(x_0)$, we also have $x_m \to x_0$. In reflexive spaces (such as $\Bbb{R}^n$), closed, bounded, non-empty convex sets are the closed convex hull of their strongly-exposed points.

(Fun fact: Spaces for which the Krein-Milman theorem holds are said to have the Krein-Milman Property. The property of the above holding in a given space is equivalent to the Radon-Nikodym property. It is an open problem as to whether these properties are different! That is, nobody knows if, given a space for which extreme points generate convex sets, it is also true that strongly-exposed points generate convex sets.)

Equivalently, we are assuming that, given any $\varepsilon > 0$, there is some $\delta > 0$ such that $$\operatorname{diam}\{x \in D : f(x) \ge f(x_0) - \delta\} < \varepsilon,$$ which is to say the slices of $D$ made with $f$ have null-convergent diameter as they become thinner.

I claim that $L^{**}(x_0) = L(x_0)$ for such a point $x_0$. Let's assume, for the sake of contradiction, that $L^{**}(x_0) < L(x_0)$. Let $$\varepsilon = \frac{L(x_0) - L^{**}(x_0)}{2} > 0.$$ By the continuity of $L$, there exists some $\delta > 0$ such that $$\|x - x_0\| < \delta \implies |L(x) - L(x_0)| < \varepsilon \implies L(x) > L^{**}(x_0) + \varepsilon.$$ Using the fact that $x_0$ is strongly exposed by $f$, there must exist some $\eta > 0$ such that $$\operatorname{diam}\{x \in D : f(x) \ge f(x_0) - \eta\} < \delta.$$ As such, if $x \in D$ such that $f(x) \ge f(x_0) - \eta$, then $x \in B(x_0; \delta)$, and hence $L(x) > L^{**}(x_0) + \varepsilon$.

Let $m \in \Bbb{R}$ be the minimum value of $L$. Define $g \in (\Bbb{R}^n \times \Bbb{R})^*$ by $$g(p, r) = (L^{**}(x_0) + \varepsilon - m)f(p) - \eta r.$$ Further, let $$K = g(x_0, L^{**}(x_0) + \varepsilon) = (L^{**}(x_0) + \varepsilon)(f(x_0) - \eta) - m f(x_0).$$ I claim that, if $(x, y) \in \operatorname{Epi} L$, then $g(x, y) \le K$. From our previous working, we have two cases:

  • $f(x) < f(x_0) - \eta$, or
  • $L(x) > L^{**}(x_0) + \varepsilon$.

In the former case, \begin{align*} K - g(x, y) &= (L^{**}(x_0) + \varepsilon)(f(x_0) - \eta) - m f(x_0) - (L^{**}(x_0) + \varepsilon - m)f(x) + \eta y \\ &> (L^{**}(x_0) + \varepsilon)(f(x_0) - \eta) - m f(x_0) - (L^{**}(x_0) + \varepsilon - m)(f(x_0) - \eta) + \eta y \\ &= \eta(y - m) \ge \eta(L(x) - m) \ge 0. \end{align*}

In the latter case, \begin{align*} K - g(x, y) &= (L^{**}(x_0) + \varepsilon)(f(x_0) - \eta) - m f(x_0) - (L^{**}(x_0) + \varepsilon - m)f(x) + \eta y \\ &= (L^{**}(x_0) + \varepsilon - m)(f(x_0) - f(x)) + \eta (y - L^{**}(x_0) - \varepsilon) \\ &\ge (L(x) - m)(f(x_0) - f(x)) + \eta (L(x) - L^{**}(x_0) - \varepsilon) \\ &\ge 0. \end{align*} On the other hand, I also claim that $g(x_0, L^{**}(x_0)) > K$: \begin{align*} g(x_0, L^{**}(x_0)) - K &= (L^{**}(x_0) + \varepsilon - m)f(x_0) - \eta L^{**}(x_0) - (L^{**}(x_0) + \varepsilon)(f(x_0) - \eta) + m f(x_0) \\ &= \eta\varepsilon > 0. \end{align*} What does this mean? It means that $(x_0, L^{**}(x_0))$ is strictly separated by a hyperplane from $\operatorname{Epi} L$, hence $$(x_0, L^{**}(x_0)) \notin \operatorname{\overline{conv}} \operatorname{Epi} L = \operatorname{Epi}(L^{**}),$$ a contradiction. Thus, $L^{**}(x_0) = L(x_0)$, i.e. $(x_0, L(x_0))$ is a support point.

Because $D$ is the closed convex hull of its strongly-exposed points, we can see that each point in $D$ (including, but not limited to, interior points) is in the closed convex hull of $x$-projections of support points.


Now, I'm aware that you didn't want just the closed convex hull. Fortunately, due to finite dimensions, we can do slightly better. Suppose $x$ lies in the closure of the strongly-exposed points of $D$. Then $(x, L(x))$ is a support point, using the continuity of $L$ and $L^{**}$. Note that the closure of these points forms a compact set, and by Caratheodory's theorem, the convex hull (not necessarily closed) is compact. Thus, we can express any point in $D$ as a convex combination (with at most $n+1$ terms) of elements of the closure of the strongly exposed points of $D$, each of which are $x$-projections of support points of $L$.

Related Question