I am working on a optimization-related research problem and need to know if the variance of a convex function is convex. I know this can be a little vague so I'm including a (rather formal) explanation below.
Say I have a function $v_i(x)$ where $v_i: \mathbb{R}^n \rightarrow \mathbb{R}$.
Assume that for each $i = 1…s$ the function $v_i$ can be a different convex function. Also, assume that the function has the form $v_i$ with probability $p_i$ where $\sum_{i=1}^{s}{p_i} = 1$.
Define the *expectation function as:
$E[v(x)] = \sum_{i=1}^{s}{p_i v_i(x)}$
And the *variance function as:
$Var[v(x)] = \sum_{i=1}^{s}{p_i*(v_i(x) – E[v(x)])^2}$
Assume the function $v_i$ is convex for all $i = 1…s$, then:
- Is the expectation function convex? (yes, right?)
- Is the variance function convex? (unsure)
Also, if $v_i$ is affine for all $i = 1…s$, then:
- Is the expectation function convex? (yes, right?)
- Is the variance function of a convex function convex? (if not, then what is it?)
Best Answer
First, I think your notation is bad. After taking expectation/variance, there should be no $i$ dependence. Also, your variance equation has one of the open parenthesis at a wrong place. It should be $$Var[v(x)] = \sum_{i=1}^{s}{p_i*(v_i(x) - E[v(x)])^2}.$$
Anyway, the expectation, as a linear combination of convex functions, is definitely convex. But the variance is not. For example, take $s=2, p_1=p_2=1/2, v_1(x)=x^2, v_2(x)=x^4$, then the "variance" would be a polynomial $\frac{x^4}4(1-2x^2+x^4)$ which is nonnegative (of course) but not convex.
If all the $v_i$ are linear functions, then so is the expectation, hence convex. In this case, the variance is also convex, because it's a linear combination of convex functions $(v_i(x) - E[v(x)])^2$. (Note that each $(v_i(x) - E[v(x)])^2$ is convex because it's the square of a linear function.)