I am amazed that the famous book by Boyd and Vandenberghe does not define a convex function in this case, but that's where I'm at – even though it is alluded to throughout the book.
Let $A \subseteq \mathbb{R}^{n_1} \times \mathbb{R}^{n_2} \times \cdots \times \mathbb{R}^{n_m}$. Suupose $f: A \to \mathbb{R}^p$ is convex. What do we mean when we say $f$ is convex in this case?
The definition provided in Boyd and Vandenberghe is of no help:
For context, this comes up when discussing the Lagrange dual function:
I understand that the Lagrange dual function is a concave function of $(\lambda, \nu)$, but I do not understand what we mean when we say $g$ is concave! I assume this just means that $-g$ is convex, but $-g$ is a function with domain over a subset of a Cartesian product, and I do not have the definition of a convex function defined over a Cartesian product!
Best Answer
Convex functions are defined for convex domains; for a domain that happens to be a product space this is no different. Recall what it means for a set $A$ to be convex: for all $x,y\in A$ and $t\in [0,1]$ we must have $tx+(1-t)y\in A$. This presupposes $A$ has an additive and scalar multiplicative structure, i.e. $A$ is a vector space. The space $\mathbb{R}^{n_1}\times\cdots\times \mathbb{R}^{n_m} \simeq \mathbb{R}^{n_1\cdots n_m}$, or generally any direct product of vector spaces, inherits the obvious vector space structure from each of the multiplicands, and so the concept of convex domain/subset is well-defined.
If $A$ happens to be a product of convex sets $S_1\subset \mathbb{R}^m$ and $S_2\subset \mathbb{R}^p$, then it can be proved that $A$ is convex, but this is not a necessary condition (consider the unit disc, which is not a product space by itself, but it is a convex subset of the product space $\mathbb{R}^2$).
In any case, once the domain is convex the provided definition makes sense. For example, your Lagrange dual $g$ is a function with domain the entirety of $\mathbb{R}^{m+p}$ (as $\lambda$ and $\nu$ have no restrictions), which is trivially a convex subset of itself, so there are no problems. To be explicit, concavity of $g$ means that for any $\lambda_1,\lambda_2,\nu_1,\nu_2$, and $t\in [0,1]$, $$g(t\lambda_1+(1-t)\lambda_2,t\nu_1+(1-t)\nu_2) \geq t g(\lambda_1,\nu_1) + (1-t) g(\lambda_2,\nu_2)$$ which is just the given definition with $x=(\lambda_1,\nu_1), y=(\lambda_2,\nu_2)$ and the inequality sign reversed.