I'm kind of rusty in calculus.
Why is the ReLU function not differentiable at $f(0)$?
$$
f(x) =
\begin{cases}
0 & \text{if $x \leq 0$} \\
x & \text{if $x > 0$}.
\end{cases}
$$
Why ReLU function is not differentiable at 0
calculus
Related Solutions
First, note that $g'(x)=1$ for $x>1$.
On the interior of each of the intervals $(-\infty,-1)$, $[-1,1]$, and $(1,\infty)$ $g$ is differentiable since each component function is. The only question is what happens at the endpoints of these intervals.
At $x=-1$, the value of both component functions is $1$ and the derivative of both component functions is $2$. This means we get $$ \lim_{h\to0^-}\frac{g(-1+h)-g(-1)}{h}=-2\tag{1} $$ Because $x^2=-1-2x$ at $x=-1$, we can use $g(x)=-1-2x$ for the computation of $(1)$.
Furthermore, we get $$ \lim_{h\to0^+}\frac{g(-1+h)-g(-1)}{h}=-2\tag{2} $$ using $g(x)=x^2$ for the computation of $(2)$.
Since the derivatives computed in $(1)$ and $(2)$ are the same, we get that $g(x)$ is differentiable at $x=-1$.
At $x=1$, the value of both component functions is $1$, however, the derivative of $x^2$ is $2$ and the derivative of $x$ is $1$. This means that $$ \lim_{h\to0^-}\frac{g(1+h)-g(1)}{h}=2\tag{3} $$ using $g(x)=x^2$ for the computation of $(3)$.
However, we get $$ \lim_{h\to0^+}\frac{g(1+h)-g(1)}{h}=1\tag{4} $$ Because $x=x^2$ at $x=1$, we can use $g(x)=x$ for the computation of $(4)$.
Since the derivatives computed in $(3)$ and $(4)$ are different, $\lim\limits_{h\to0}\frac{g(1+h)-g(1)}{h}$ does not exist, and therefore $g(x)$ is not differentiable at $x=1$.
It it was differentiable at $0$, then it would be true that: $$ \lim_{x \to 0+} \frac{f(x)}{x} = \lim_{x \to 0-} \frac{f(x)}{x} = f'(0).$$ Above, the $x\to 0+$ means that we take the limit over the positive values of $x$, and likewise for $x \to 0-$.
Now, these two limits are easy to compute, and unfortunately are different. First, you have: $$ \lim_{x \to 0-} \frac{f(x)}{x} = 0.$$ Secondly, you get with a little more work: $$ \lim_{x \to 0+} \frac{f(x)}{x} = \lim_{x \to 0+} \frac{1}{1+x} = 1.$$ Now, these two values obviously can't be both equal to $f'(x)$. Thus, $f$ is not differentiable at $0$.
As for the latter part: NO, the derivative does not automatically have to be continuous. There is a Wikipedia article that you will surely find relevant. For example, the function $f$ given below is differentiable, but the derivative $f'$ is not continuous at $0$: $$ f(x) = \begin{cases} x^2 \sin \frac{1}{x} \quad& x > 0\\ 0 \quad& x \leq 0 \end{cases}$$ The trick is that the term $x^2$ assures that $f$ goes to $0$ fast enough to have derivative $0$ at $0$, but the term $\sin \frac{1}{x}$ assures that close to $0$ the function has a large slope.
On the other hand, the derivative always has the mean value property, which is known as Darboux Theorem.
Best Answer
Because it has a sharp corner at $0$, so it doesn't have a well defined tangent line; think about it, you can imagine many lines going through $(0, 0)$ that are tangent to the graph, so there are many possible tangent lines.
More formally, we have to investigate the limit
$$\lim_{h \to 0} \dfrac{f(0+h) - f(0)}{h}$$
This limit does not exist for the function, because if you let $h$ approach $0$ from the right, you get
$$\lim_{h \to 0^+} \dfrac {h-0}{h}=1.$$
While if you let $h$ approach $0$ from the left, that limit
$$\lim_{h \to 0^-} \dfrac {0-0}{h}=0.$$
Therefore the limit does not exist, so the function is not differentiable at $0$.