[Math] Existence of minimizer for strongly convex function on closed, convex set

convex optimizationconvex-analysisoptimization

Let $f : \mathbb{R}^n \to \mathbb{R}$ be strongly convex. Consider the problem

$$\min_{x \in A} f(x)$$

where $A \subseteq \mathbb{R}^n$ is nonempty, convex, and closed. Also assume $\inf_{x \in A} f(x) < \infty$ (if this is not the case, then $f(x) = \infty$ for all $x \in A$, which is not a particularly interesting scenario).

I know that if $f$ has a minimizer, on $A$, then it is unique (see this thread for an explanation of why). My question is are these conditions sufficient to guarantee the existence of such a minimizer? If not, what else would need to be assumed to imply this?

EDIT: Here is the definition of strongly convex (from Wikipedia): $f$ is strongly convex if there exists a constant $m > 0$ such that for all $x$ and $y$ in the domain and $\lambda \in [0,1]$ the following inequality holds:

$$f(\lambda x + (1-\lambda)y) \leq \lambda f(x) + (1-\lambda) f(y) – \frac{1}{2} m\lambda (1-\lambda) \| x – y \|^2$$

This is equivalent to the following:

$$g := f – \frac{m}{2}\|\cdot\|^2$$

is convex, where $\|\cdot\|$ is induced by an inner product space. See this paper for more on the equivalence of these definitions.

Best Answer

Thanks to those that have already responded, you were very helpful. Here I will give the solution I have come up with, with more exposition than is provided in some of the other responses.

First we need the following lemma:

Lemma: $\lim_{\|x\| \to \infty} f(x) = \infty$. Some authors refer to this as $f$ being coercive.

Proof: Let $x_0 \in \mathbb{R}^n$ and let $v$ be a subgradient of $g$ at $x_0$, i.e. $x_0 \in \partial g(x_0)$. By equivalence of norms in finite-dimensional vector spaces, there exists a constant $c > 0$ such that $\|x\|_2 \leq c \|x\|$ for all $x \in \mathbb{R}^n$. By Cauchy-Schwarz and the trinagle inequality, for $\|x\| > 0$ we have

$$ \begin{align*} \frac{| v^T(x - x_0) |}{\frac{m}{2}\|x\|^2} &\leq \frac{\|v\|_2 \|x - x_0\|_2}{\frac{m}{2}\|x\|^2} \\ &\leq \frac{\|v\|_2 \|x\|_2 + \|v\|_2 \|x_0\|_2}{\frac{m}{2}\|x\|^2} \\ &\leq \frac{c\|v\|_2 \|x\| + \|v\|_2 \|x_0\|_2}{\frac{m}{2}\|x\|^2} \\ &= \frac{2c\|v\|_2 \|x\|}{m} \frac{1}{\|x\|} + \frac{2\|v\|_2 \|x_0\|_2}{m} \frac{1}{\|x\|^2} \end{align*} $$

The far right hand side of this inequality $\to 0$ as $\|x\| \to \infty$, which implies that $v^T(x - x_0) + \frac{m}{2} \|x\|^2 \to \infty$ as $\|x\| \to \infty$. Now we use the definition of subgradient:

$$ \begin{align*} v^T(x - x_0) &\leq g(x) - g(x_0) \\ v^T(x - x_0) + \frac{m}{2}\|x\|^2 &\leq g(x) + \frac{m}{2}\|x\|^2 - g(x_0) \\ v^T(x - x_0) + \frac{m}{2}\|x\|^2 + g(x_0) &\leq f(x) \end{align*} $$

The left hand side of this $\to \infty$ as $\|x\| \to \infty$, so we conclude that $f(x) \to \infty$ as $\|x\| \to \infty$. $\square$

On to the main result. First, assume that $A$ is unbounded. If it is bounded, then it is compact, and the result follows immediately. There are 2 mutually exclusive possibilities:

Case 1: $f$ has a minimizer on $A$, in which case it is unique (see this thread).

Case 2: $f$ does not have a minimizer on $A$.

Assume we have case 2. Let $f^\star := \inf_{x \in A} f(x)$. $f^\star < \infty$ by assumption. Let $(x_k)$ be a sequence in $A$ such that $f(x_k) \to f^\star$. We now have two mutually exclusive subcases:

Subcase 2.1: $\sup_k \|x_k\| = d < \infty$. Define $B_d := \{ x \in \mathbb{R}^n \ : \ \|x\| \leq d\}$. Then for all $k$, $x_k \in \{ A \cap B_d \}$ which is closed and bounded and hence compact. Therefore $(x_k)$ converges in $A$, i.e. $x_k \to x^\star$ for some $x^\star \in A$. Continuity of $f$ then implies $f(x^\star) = f^\star$, which is a contradiction.

Subcase 2.2: $\sup_k \|x_k\| = \infty$. This implies $\|x_k\| \to \infty$, and by the Lemma this implies $f(x_k) \to \infty$ which implies $f^\star = \infty$ which contradicts $f^\star < \infty$.

Thus we conclude that Case 2 cannot occur, and therefore Case 1 must occur.

EDIT: After writing all of this out, it is clear that $f$ strongly convex is a stronger assumption than we require. $f$ strictly convex and coercive is sufficient for $f$ to have a unique global minimum on the convex set $A$.