Let $f(x,y) = y^3+3xy^2 -6y^2 +2$ .Find the extreme values and the saddle points of $f$ if exist .
My try : I used second derivatives test . The critical points are $(a,0), a\in \mathbb{R}$ . Therefore the value of $D$ is always zero at the critical points and the test is inconclusive. I don't know what is the next step .
Best Answer
Indeed, the critical points are $(a,0)$ with $a \in \mathbb R$, just as you have found. Indeed, the second derivative test is not usable in this problem, so we'll have to study the behaviour of $f$ around the points $(a,0)$ "by hand" (i.e. by improvising).
If $a < 2$, choose $r>0$ such that $4r < 6 - 3a$. Consider the neighbourhood $(a-r,a+r) \times (-r,r)$ of $(a,0)$ and notice that on it
$$y + 3x - 6 < r + 3(a+r) - 6 = 4r + 3a - 6 < 0$$
so that
$$f(x,y) - f(a,0) = y^2(y + 3x - 6) \le y^2 (4r + 3a - 6) \le 0 \ ,$$
which shows that $f(x,y) \le f(a,0)$ on this neighbourhood, so $(a,0)$ is a point of local maximum.
If $a>2$ the analysis is similar: you choose $0 < r < 3a - 6$ and you get that $(a,0)$ is a point of local minimum.
Finally, if $a=2$ then again choose a neighbourhood $(2-r,2+r) \times (-r,r)$ of $(2,0)$ (this time without imposing any restriction on $r>0$) and notice that:
for $x \in (2-r, 2)$ and $y \in (-r ,0)$ you have $y + 3x - 6 < 0 + 3 \cdot 2 - 6 = 0$, so that $y^2 (y + 3x - 6) <0$;
for $x \in (2, 2+r)$ and $y \in (0,r)$ you have $y + 3x - 6 > 0 + 3 \cdot 2 - 6 = 0$, so that $y^2 (y + 3x - 6) > 0$.
This shows that on this neighbourhood of $(2,0)$ the difference $f(x,y) - f(2,0)$ takes both negative and positive values, therefore $(2,0)$ is a saddle point.