This question is related to answer for
How do I prove that objective function is not convex.
The objective expression becomes $2y^2x^2$. The Hessian of this expression is
$$\begin{bmatrix} 4y^2 & 8xy \\ 8xy & 4x^2 \end{bmatrix}$$
The determinant of this Hessian is $-48x^2y^2$, which is negative when both $x$ and $y$ are nonzero, so it cannot possibly be positive semidefinite. Hence the objective function is neither convex nor concave.
I don't understand the last line of this answer where author says "Hence the objective function can is NEITHER CONVEX nor CONCAVE."
In other words, can someone fill in these blanks ?
If determinant of hessian matrix of $\mathbf{A}$ is
- POSITIVE, $\mathbf{A}\in S^{N}_{++}$ and objective function is CONVEX
- ZERO, $\mathbf{A} \in$ ___ and objective function is _________
- NEGATIVE, $\mathbf{A} \in$ ___ and objective function is _________
Best Answer
Hint for 3: What is the opposite of convex?
Hint for 2: If a point is neither convex nor concave, it must be convex in one direction and concave in another (or flat).