As you say, $f \,:\, A \to \mathbb{R}$ with $A \subset \mathbb{R}$ means that for every $x \in A$, the derivative $f'(x)$ exists. In other words, for every $x \in A$, the limit $$
f'(x) = \lim_{y \to x} \frac{f(y) - f(x)}{y-x}
$$
exists. Note that, due to the way limits works, whether or not $f'(x)$ exists depends only on the behaviour of $f$ in the immediate vicinity of $x$. More formally, if you have two functions $f$ and $g$, and some $\epsilon$ such that $f(y) = g(y)$ for all $y \in (x-\epsilon,x+\epsilon)$, then $f'(x) = g'(x)$.
Since you probably already know that the derivatives of $x \to x^2$ and $x \to -x^2$ exist on the whole real line, you therefore know that the derivative of $$
f(x) = \begin{cases} x^2 &\text{if $x\geq 0$} \\ -x^2 &\text{if $x < 0$} \end{cases}
$$
exists everywhere except at $x=0$. Because if $x \neq 0$, then $f(x) = x^2$ or $f(x) = -x^2$ on some small interval around $x$.
So all you have to do is to decide whether or not the derivate at $x=0$ exists or not. The first step is to check that $f$ is continuous at $x=0$. Since both $x^2$ and $-x^2$ are continuous $0$, it suffices to check that they take the same value at $x=0$ - and they do, $0^2 = -0^2$ after all. Note that being continuous at a point is a necessary (but not sufficient!) condition for being differentiable at that point. So had the continuity check failed, you could have immediately concluded that $f$ is not differentiable at $0$.
Since $f$ is continuous at $x=0$, and since the derivative of both $x^2$ and $-x^2$ also exists at $x=0$, it's sufficient for the derivative of $x^2$ and $-x^2$ to also take the same value at $x=0$ for $f$ to be differentiable there. And, indeed, they do - $\frac{d}{dx}(x^2) = 2x$ and $\frac{d}{dx}(-x^2) = -2x$, which at zero both take the value $0$. Thus, your $f$ is indeed differentiable everywhere.
$$f (2^-)=\lim_{x\to 2^-} (\frac {x}{2}+2)=1+2=3$$
$$ f (2^+)=\lim_{2^+}\sqrt {2x}=2$$
$f $ is not continuous at $x=2$ thus it is not differentiable at $x=2$.
By definition, differentiable at $x=x_0$ means
$$\exists L\in \mathbb R \;\exists \eta>0 :\forall x\in (2-\eta,2+\eta) $$
$$f (x)=f (2)+(x-2)\Bigl (L+\epsilon (x)\Bigr) $$
with $$\lim_{x\to 2}\epsilon (x)=0$$
Best Answer
Suppose $f$ is continuous at $0$. Without this assumption, user math_man's answer shows that the statement is not true.
We need to argue that $\displaystyle f'(0)=\lim_{h\to 0}\frac{f(h)-f(0)}h$ exists. But this is now immediate from L'Hôpital's rule: The assumptions that $f$ is continuous at $0$ (and therefore everywhere), and differentiable away from $0$ give us that we can apply the mean-value theorem. This allows us to conclude that for any $h$ there is a $\xi_h$ in between $0$ and $h$ such that $$ \frac{f(h)-f(0)}h=\frac{f'(\xi_h)}1=f'(\xi_h). $$ Now note that $\xi_h\to0$ as $h\to 0$. Since we are given that $\lim_{t\to0}f'(t)=0$, it follows that $\lim_{h\to0}f'(\xi_h)=0$ as well, so $f'(0)$ exists and is $0$.