As in the title I found critical points to be $(1,0)$-saddle point, $(-1,0)$– local minimum and $(0,0)$ is the one I have problem with.
Second derivative test is inconclusive in this case. If it was a saddle point I should find some curve in the graph that has inflection point at $(0,0)$ to prove it. Unfortunately, I suspect it to be local maximum and in this case I have no clue how can I prove that it grows in neighborhood of $(0,0)$ in every direction.
[Math] Find and classify critical points of $x^4-y^4-4xy^2-2x^2$, second derivative test is inconclusive.
calculusmultivariable-calculusoptimizationreal-analysis
Related Solutions
Hint: what can you say about the fourth power of any real number?
Saddle point for sure. The Hessian of $f$ is: $$ H_f = \begin{pmatrix} 2x(2x^2-3) y e^{-x^2-y^2} & (2x^2-1)(2y^2-1) e^{-x^2-y^2} \\ (2x^2-1)(2y^2-1) e^{-x^2-y^2} & 2y(2y^2-3) x e^{-x^2-y^2} \end{pmatrix}, $$ thus $$ H_f(0,0) = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}, $$ and $\mathrm{det}(H_f(0,0)) = -1$ not $1$, which shows $(0,0)$ is a saddle point.
Now to answer your first question.
It is impossible for a smooth function that can have $\mathrm{det}(H_f(x_0,y_0)) > 0$ and $f_{xx}(x_0,y_0) = 0$ at $(x_0,y_0)$.
Let's say if $f_{xx}(x_0,y_0) = 0$, then $$ H_f(x_0,y_0) = \begin{pmatrix} 0 & f_{xy}(x_0,y_0) \\ f_{yx}(x_0,y_0) & f_{yy}(x_0,y_0) \end{pmatrix}. $$ Unless you construct some special smooth functions that bear the property that $f_{xy} \neq f_{yx}$, the determinant $\mathrm{det}(H_f(x_0,y_0)) = - (f_{xy}(x_0,y_0))^2 \leq 0$.
For the second question: when the test is inconclusive,
$f_{xx}$ and $f_{yy}$ have different signs, then we have a saddle point. You can look at the picture to see the geometric meaning of this (though in example they are zero): If our viewponint is somewhere on the $y$-axis and to observe the change in $x$, $f_{xx}>0$ means what we see is a concave up curve near the neighborhood of the point of interest $(x_0,y_0)$; Moving our viewponint to somewhere on the $x$-axis and to observe the change in $y$, $f_{yy}<0$ means what we see is a concave down curve. Then, clearly at $(x_0,y_0)$ we have a saddle.
$f_{xx}$ and $f_{yy}$ have the same signs, then we have a local maximum/minimum.
Best Answer
Consider the following graphs: \begin{align} y = x & \implies x^4-y^4-4xy^2-2x^2= -4x^3-2x^2\\ &\implies x=0\text{ is a local maximum along the curve}\\ y^2 = (-2\pm \sqrt{2})x, x<0 & \implies x^4-y^4-4xy^2-2x^2= x^4\\ &\implies x=0\text{ is a local minimum along the curve} \end{align} Hence, $(0,0)$ is a saddle point.