I should analyze whether the function
$$f(x,y) = x^2y + x y^2 \text{ where } x,y > 0$$
is (quasi-) concave or convex.
Thus, as usual, I set up the Hessian as
$$ D^2f(x,y) = \left( \begin{array}{cc}
2y & 2x + 2y \\
2x + 2y & 2x \\
\end{array} \right) $$
which, given the constraints $x,y > 0$, is indefinite.
So I need the bordered Hessian:
$$
H= \left( \begin{array}{ccc}
0 & 2xy + y^2 & x^2 + 2xy \\
2xy + y^2 & 2y & 2x + 2y \\
x^2 + 2xy & 2x+2y & 2x \end{array} \right) $$
and check its determinant to find out about its properties. I end up with
$$ \det|H| = 2x(2x^3 + x^3 y – 2x^2 y + 4x^2 y^2 + 4 x y^3 + 2 y^4)$$
which, considering $x,y > 0$, looks quite positive to me, so $f$ would be quasiconcave.
But how can I prove that $\det|H| > 0$? Or is there any easier approach?
Thanks!
Best Answer
The Hessian $D_2$ of your function $f$ is as you said, then: $$ D_2 - \lambda I = \begin{pmatrix} 2y - \lambda & 2x+2y \\ 2x+2y & 2x-\lambda \end{pmatrix}. $$ The determinant is: $$ \det(D_2-\lambda I) = \lambda^2 - 2(x+y)\lambda - 4(x^2+y^2+xy) , $$ hence we have one positive and one negative eigenvalues, surely it is not positive semi-definite.
The bordered Hessian $H$ has determinant (I don't know what your expression in the question is, but it sure doesn't look right): $$ \det (H) = 6 x^4 y + 12 x^3 y^2 + 12 x^2 y^3 + 6 x y^4 >0 \quad \forall x,y>0. $$
Another way to do this is considering the level set
$$F_a := \{(x,y): f(x,y)\ge t\}$$ is convex or not:
as you can tell by the level curve the level set is indeed convex, hence $f$ is quasi-concave. To see it mathematically: set $$ x^2y + xy^2 -t =0 \implies y = -\frac{x}{2} + \sqrt{\frac{x^2}{4} + \frac{t}{x}}, $$ and $$ y'' = \frac{6 t (t+x^3)}{x^{5/2} (4 t+x^3)^{3/2}} >0 \quad \forall t>0, $$ therefore $F_a$ is convex and the quasi-concavity of $f$ follows.