The log-sum-exp function $f: \; \mathbb R^n \to \mathbb R$ is defined by
$$f(x)=\ln \left (e^{x_1}+\cdots + e^{x_n} \right).$$
It is well-known that this function is convex, but I wonder that whether or not $f$ is strictly convex?
Thank you for any answer.
Best Answer
No, it is not. Consider the case when $x_1 =\dots=x_n=y$. Then $f(x)=y+\log n$. The function is affine along this line, so it is not strictly convex.
In fact, it is affine along any line of the form $x = y \vec{1} + b$, where $b$ is constant: $f(x) = y + \log\sum_i e^{b_i}$.
EDIT: Richkent wants to know if there are any other lines along which $f$ is affine. The answer is no. To see why, let's look at the Hessian of $f(x)$, which has a special structure: $$\nabla f(x) = g, \quad \nabla^2 f(x) = \mathop{\textrm{diag}}(g) - gg^T, \quad g \triangleq \frac{1}{\sum_i e^{x_i}} \begin{bmatrix} e^{x_1} \\ \vdots \\ e^{x_n} \end{bmatrix}$$ Notice that $g\succ 0$ and $\vec{1}^Tg = 1$, so $\vec{1}^T \nabla^2f(x) \vec{1} = 0$. This confirms that $f$ is affine along the directions $v=\alpha\vec{1}$.
But note also that $\nabla^2 f$ is a rank-1 modification of a positive definite matrix $\mathop{\textrm{diag}}(g)$. This is important, because it tells us that $\mathop{\textrm{rank}}(\nabla^2 f(x))=n-1$, and therefore that $v=\alpha\vec{1}$ are the only vectors for which $v^T\nabla^2 f(x) v=0$. Thus $f$ is strictly convex along any other directions.
To see this a bit better, let's look at the function $g(t)=f(x+tv)$, where $(x,v)$ are fixed. This is just a slice of $f$ along the line $x+tv$, so of course it is convex. The second derivative of $g$ is $$g''(t) = v^T \nabla^2 f(x+tv) v.$$ If $v=\alpha\vec{1}$, then $$g''(t) = v^T\nabla^2 f(x+tv) v=\alpha^2\vec{1}^T\nabla^2(x+tv)\vec{1}=0$$ confirming that $g$ is affine if $v=\alpha\vec{1}$. But if $v\neq\alpha\vec{1}$, then $$g''(t) = v^T\nabla^2 f(x+tv) v>0$$ which means that $g$ is strictly convex.