Let $x=r\cos \theta, \quad y=r\sin \theta$
Therefore $$|\frac{xy}{\sqrt{x^2+y^2}}|=r |\cos \theta \sin \theta|\le r=\sqrt{x^2+y^2}\lt \epsilon$$if $x^2\lt\frac{\epsilon^2}{2},\quad$and$\quad y^2\lt \frac{\epsilon^2}{2}$
or if $|x|\lt\frac{\epsilon}{\sqrt2},\quad$and$\quad |y|\lt \frac{\epsilon}{\sqrt2}$
Thus $$|\frac{xy}{\sqrt{x^2+y^2}}-0|\lt \epsilon,\qquad \text{where}\quad |x|\lt\frac{\epsilon}{\sqrt2},\quad and \quad |y|\lt \frac{\epsilon}{\sqrt2}$$
$$\implies \lim_{(x,y)\to (0,0)}\frac{xy}{\sqrt{x^2+y^2}}=0$$
Hence $$\lim_{(x,y)\to (0,0)}f(x,y)=f(0,0)$$and therefore $f(x,y)$ is continuous at $(0,0)$
Again $$f_x(0,0)=\lim_{h \to 0}\frac{f(h,0)-f(0,0)}{h}=0$$and $$f_y(0,0)=\lim_{k \to 0}\frac{f(0,k)-f(0,0)}{k}=0$$
Thus the function $f(x,y)$ possesses partial derivatives at $(0,0)$.
If the given function is differentiable at $(0,0)$, then by definition $$df=f(h,k)-f(0,0)=Ah+Bk+h\phi +k\psi\qquad . . . . . (1)$$ where $\quad A=f_x(0,0)=0;\quad B=f_y(0,0)=0, \quad $and $~\phi,~\psi~$ tends to zero as $\quad (h,k)\to (0,0)$.
So from $(1)$ we have $$\frac{hk}{\sqrt{h^2+k^2}}=h\phi +k\psi$$
Putting $\quad k=mh \quad $and letting $\quad h\to 0,\quad$ we have
$$\frac{m}{\sqrt{1+m^2}}=\lim_{h \to 0}(\phi +m\psi)=0$$which is impossible for arbitraty $~m$.
Hence the function is not differentiable at $(0,0)$.
To complement user's answer, I would like to point out that the example in the OP is even more striking since not only do partial derivatives $\partial_1f(0,0)$ and $\partial_2f(0,0)$ exists, but also the directional derivative of the function $f$ at $\boldsymbol{0}=(0,0)$ along any direction $\mathbf{v}=(h,k)$ exists:
$$\partial_\mathbf{v}f(0,0):=\lim_{t\rightarrow0}\frac{f(\boldsymbol{0}+t\mathbf{v})-f(\boldsymbol{0})}{t}=\lim_{t\rightarrow0}\frac{1}{t}\frac{t^3k(h^2+k^2)}{t^2(h^2+k^2)}=k$$
So to add to other solutions:
A function $f$ may be
- continuous at some point $\mathbf{c}$,
- have (finite) directional derivatives at $\mathbf{c}$ along any vector $\mathbf{v}$ ($\partial_1f(\mathbf{c})$ and $\partial_2f(\mathbf{c})$ correspond to $\mathbf{v}=(1,0)$ and $\mathbf{v}=(0,1)$ respectively)
and yet not be differentiable.
Best Answer
Hint: (i) Approach $(0,0)$ along the curve $x=y^2$; (ii) Approach $(0,0)$ along the curve $x=y$.