[Math] Finding extreme values when the determinant of the Hessian at a critical point is zero.

multivariable-calculus

We want to determine extreme values of $f(x,y)=x^3+xy^2-x^2y-y^3$.
We first determine critical points by solving $\dfrac{\partial f(x,y)}{\partial x}=0$ and
$\dfrac{\partial f(x,y)}{\partial y}=0$ which gives that the only critical point is $(0,0)$.
Now we compute the determinant of the Hessian $$D(x,y)=(6x-2y)(2x-6y)-(2y-2x)^2$$
Hence $D(0,0)=0$ and the determinant of the Hessian test is not conclusive, so what to do next to verify existence of local and global extreme values? thank you for your help.

Best Answer

A hint:

You can factor your function and decide what's happening at $(0,0)$ by inspection.