Let us start with
$$h(x) = \begin{cases}
\hphantom{-}4(x+2) &, -2\leqslant x < -\frac{3}{2}\\
-4(x+1) &, -\frac{3}{2} \leqslant x < -1\\
-4(x-1) &,
\hphantom{-}\; 1 \leqslant x < \frac{3}{2}\\
\hphantom{-} 4(x-2) &,
\hphantom{-}\frac{3}{2}\leqslant x < 2\\
\qquad 0 &,
\hphantom{-} \text{ otherwise.} \end{cases}$$
For $c > 0$, let
$$h_c(x) = c\cdot h(c\cdot x).$$
Then $h_c$ is continuous, and $\int_{-\infty}^0 h_c(x)\,dx = 1$ as well as $\int_{-\infty}^\infty h_c(x)\,dx = 0$. Now let
$$g(x) = \sum_{n=1}^\infty h_{5^n}\left(x-n-\frac{1}{2}\right).$$
Every $g_n(x) = h_{5^n}\left(x-n-\frac12\right)$ vanishes identically outside the interval $[n,n+1]$, so $g$ is continuous, and
$$f(x) = \int_0^x g(t)\,dt$$
is well-defined and continuously differentiable.
Furthermore, $f(x) \equiv 0$ on every interval $\left[n, n+\frac{1}{2} - \frac{2}{5^n}\right]$, and $f(x) \equiv 1$ on every interval $\left[n+\frac{1}{2}-\frac{1}{5^n}, n+\frac{1}{2}+\frac{1}{5^n}\right]$. Thus on
$$X = \bigcup_{n=1}^\infty \left(\left[n, n+\frac{1}{2} - \frac{2}{5^n}\right] \cup \left[n+\frac{1}{2}-\frac{1}{5^n}, n+\frac{1}{2}+\frac{1}{5^n}\right]\right)$$
we have $f' \equiv 0$, so the derivative is bounded, but
$$f\left(n+\frac{1}{2}-\frac{1}{5^n}\right) - f\left(n+\frac{1}{2}-\frac{2}{5^n}\right) = 1$$
for all $n$, while the distance between the two points is $5^{-n}$ which becomes arbitrarily small, so $f$ is not uniformly continuous on $X$.
If the sentence
Note that $X$ cannot be an interval, a disjoint union of intervals, nor can it be a discrete set.
was meant to forbid a construction as above where $X$ is a disjoint union of intervals, we can obey the letter of the law (but not the spirit) by adding an arbitrary subset of $(-\infty,0)$ that is not a union of disjoint intervals.
Computing the derivative from the left gives you
$$
\lim\limits_{h\rightarrow0^-} {f(0+h)-f(0)\over h }
=\lim\limits_{h\rightarrow0^-} {h-1\over h }=\infty.
$$
(In particular, note $f(0)=1$, not $0$.)
You can also see the derivative from the left doesn't exist (as a real number) by considering slopes of secant lines. Note a secant line has one endpoint at the point $(0,1)$ and the other at a point $(h,h)$ with $h<0$. As $h$ tends to $0$, the slopes tend to $\infty$.
Best Answer
Yes:
Start with the standard $$h(t)=\begin{cases}t^2\sin(1/t),&(t\ne0), \\0,&(t=0).\end{cases}$$ So $h$ is differentiable and $h'$ is continuous except at $0$. Since $h'$ is locally bounded there exists a differentiable function $g$ with $g(t)=h(t)$ for $|t|\le1$ and such that $g$ and $g'$ are bounded.
Say $\Bbb Q=\{r_1,r_2,\dots\}$. Let $$f(t)=\sum 2^{-n}g(t-r_n).$$It follows that $f$ is differentiable and $$f'(t)=\sum 2^{-n}g'(t-r_n),$$since the last sum is uniformly convergent (cf. baby Rudin Thm 7.17.). It's clear that $f'$ is continuous at $t$ if $t$ is irrational, again by uniform convergence.
And $f'$ is discontinuous at $t$ if $t$ is rational. Details for that: Say $t=r_n$. Write $$f=f_1+f_2,$$where $$f_1(t)=2^{-n}g(t-r_n).$$Then as above, uniform convergence shows that $f_2'$ is continuous at $r_n$; since $f_1'$ is discontinuous there so is $f$.
Note
No, the Thomae function $f$ does not have an antiderivative. But there's a major gap in the explanation for this in various comments: It's clear that if $g(y)-g(x)=\int_x^yf$ then $g$ is constant, hence $g'\ne f$. But it's not clear why $g'=f$ would imply $g(x)-g(y)=\int_y^x f$, since after all $f$ is not continuous. Possibly one could justify this using some fancy version of FTC.
Edit. In fact it's easy to show that if $g$ is differentiable and $g'$ is Riemann integrable then $g(x)-g(y)=\int_y^x g'$; I was forgetting this. So the argument in those comments is fine, although probably someone might have mentioned the bit about Riemann integrability.
Anyway, there's a simple argument without FTC:
The point being that although a derivative need not be continuous, it can't be "too discontinuous". For example it's well known that a derivative cannot have a jump discontinuity. That's not quite enough here, but:
Proof: It's an easy exercise from the definitions to show there exists a sequence $t_n$ decreasing to zero such that $$\frac{g(t_n)-g(t_{n+1})}{t_n-t_{n+1}}\to g'(0).$$So MVT shows here exists a sequence $s_n\to0$ (with $s_n>0$) such that $$g'(s_n)\to g'(0).$$
Otoh if $f$ is the Thomae function then $$\limsup_{t\to0}f(t)<f(0).$$So the lemma shows that $f$ is not a derivative.