Being complex differentiable at a point is equivalent to the combination of
- Being real differentiable at that point, and
- Satisfying the Cauchy-Riemann equations
The real and imaginary parts of $f$ are $u=x^2$ and $v= ^2$. They are polynomials, so real-differentiable everywhere. The two Cauchy-Riemann equations take the form $2x=2y$ (from $u_x=v_y$) and $0=0$ (from $ u_y=-v_x$). The second holds everywhere. The first holds when $x=y$ and only then.
Outside of the origin, it is convenient to write $g$ in a different manner. The real and imaginary part of the numerator have the common factor $xy$, and taking that out, we can write
$$g(x+iy) = \begin{cases} \dfrac{xy}{x^2+y^2}(x+iy) &, (x,y) \neq (0,0) \\ \qquad 0 &, (x,y) = (0,0). \end{cases}\tag{1}$$
In that form, the answer to both questions is easily found by looking at the factor $\frac{xy}{x^2+y^2}$. The inequality between arithmetic and geometric mean shows
$$\lvert xy\rvert \leqslant \frac{x^2+y^2}{2},$$
and hence $\Bigl\lvert \frac{xy}{x^2+y^2}\Bigr\rvert \leqslant \frac{1}{2}$. This shows
$$\lvert g(x+iy)\rvert \leqslant \frac{1}{2}\lvert x+iy\rvert$$
for $x+iy \neq 0$, from which the continuity of $g$ at $0$ follows easily.
For the question of complex differentiability, $(1)$ lets us write the difference quotient
$$\frac{g(x+iy) - g(0)}{x+iy} = \frac{xy}{x^2+y^2},\tag{2}$$
and one notes that the right hand side has no limit as $(x,y) \to (0,0)$, every value in $\bigl[-\frac{1}{2},\frac{1}{2}\bigr]$ is attained on every circle $\{x+iy : \sqrt{x^2+y^2} = \rho\}$. On the ray $\{ r(\cos \varphi + i\sin\varphi) : r \in (0,+\infty)\}$ the difference quotient has the constant value $\frac{1}{2}\sin (2\varphi)$. Thus $g$ is not complex differentiable at $0$.
This function $g$ shows that it is not sufficient to imply complex differentiability that the partial derivatives exist and satisfy the Cauchy-Riemann equations, (total) real differentiability (Fréchet differentiability) is required.
Best Answer
$$\begin{pmatrix} \frac{\partial u}{\partial x}&\frac{\partial u}{\partial y}\\ -\frac{\partial u}{\partial y}&\frac{\partial u}{\partial x}\\ \end{pmatrix}\begin{pmatrix} x\\y\\ \end{pmatrix} =\begin{pmatrix} \frac{\partial u}{\partial x}x +\frac{\partial u}{\partial y}y\\ -\frac{\partial u}{\partial y}x+\frac{\partial u}{\partial x}y\\ \end{pmatrix}$$
On the other hand,
$$\left(\frac{\partial u}{\partial x}-i\frac{\partial u}{\partial y}\right) (x+iy)= \frac{\partial u}{\partial x}x +\frac{\partial u}{\partial y}y+ \left(-\frac{\partial u}{\partial y}x+\frac{\partial u}{\partial x} y\right)i$$
Compare the terms and see they are the same.