Lebesgue's criterion for Riemann integrability states that a function $f: [a, b] \longrightarrow \Bbb R$ is Riemann integrable if and only if it is bounded and continuous almost everywhere on $[a,b]$.
In the present case, $|F(x)| \leq |b| \sqrt{|b|}$ on $[0, |b|]$ and $F$ is also continuous on $[0, |b|]$, so by Lebesgue's criterion $F$ is Riemann integrable.
On the other hand, $F'$ is not bounded in any neighborhood of $x = 0$, so by Lebesgue's criterion it is not Riemann integrable.
The point of this example is that you can't always integrate a function just because that function is a derivative of some other function.
To show $F$ is continuous at $c$:
We know that $f$ is Riemann-integrable because $f$ has only one discontinuity on $[a,b]$, and that discontinuity is a jump discontinuity (the one-sided limits of $f$ as $f$ approaches $c$ are finite), so $f$ is bounded on $[a,b]$. So by Lebesgue's criterion of Riemann integrability, $f$ is Riemann integrable on $[a,b]$.
Now $$F(x)-F(c)=\int_a^xf(t) \, dt-\int_a^cf(t) \, dt = \int_c^x f(t)\, dt$$
Since $f$ is bounded, for all $x \in [a,b]$, $|f(x)|\leq M$, for some $M \in \mathbb{R}$. Therefore: $$|\int_c^xf(t) \, dt| \leq \int_c^x|f(t)| \, dt \leq \int_c^xM \, dt$$
$$\implies -\int_c^x M\;dt\leq \int_c^x f(t)dt\leq \int_c^xM \;dt$$
$$\implies -M(x-c)\leq \int_c^x f(t)dt\leq M(x-c)$$
Taking the limit as $x$ tends to $ c$, the squeeze theorem says $\lim\limits_{x\to c}F(x)=F(c)$. So $F$ is continuous at $c$.
$\blacksquare$
To show that $F$ is not differentiable at $c$:
With help from Showing that an indefinite integral of a function with a jump discontinuity is not differentiable at the jump discontinuity
Lemma: The left derivative of $F(x)$ at $c$ is $\lim_\limits{x\to c^-} f(x)$ and the right derivative of $F(x)$ at $c$ is $\lim_\limits{x\to c^+} f(x)$.
Proof: Define $g:[a,c]\to \mathbb{R} $ by:
$$g(x) = \begin{cases}
f(x) & \text{if } x<c \\ \lim_\limits{x \to c^-} f(x) & \text{if } x = c
\end{cases}
$$
Then $g$ is continuous on $[a,c]$.
Now for all $x \in [a,c]$, $F(x) = \int_a^x f(t) \,dt =\int_a^x g(t) \,dt $. Why? Well if we interpret the Riemann integral as a Lebesgue integral, this is very easy since $f=g$ almost everywhere on $[a,c]$. If you want all the technical details with Riemann integrals see here: Proving Riemann integral does not change when finite values of a function is changed.
Now we show that the continuity of $g$ implies that the left derivative of $F$ at $c$ is $g(c)$.
We note that $g(c)$ is a constant, so $(x-c)g(c) = \int_c^x g(c) dt$, hence $g(c) = \frac{\int_c^x g(c) dt}{x-c}$.
Then given $x \neq c$,
$$|\frac{F(x)-F(c)}{x-c} - g(c)|$$
$$= |\frac{\int_a^x g(t) \, dt - \int_a^c g(t) \, dt}{x-c} - \frac{\int_c^x g(c) dt}{x-c}| $$
$$= |\frac{\int_c^x g(t) \, dt}{x-c} - \frac{\int_c^x g(c) dt}{x-c}| $$
$$= \frac{1}{|x-c|} |\int_c^x (g(t) - g(c)) \, dt |$$
$$\leq \frac{1}{|x-c|} |\int_c^x |g(t) - g(c)| \, dt |$$
Since $g$ is continuous at $c$, there exists a $\delta>0$ such that for all $x \in [a,c]$, if $|x-c|<\delta$, then $|g(x)-g(c)|<\epsilon$.
So take any $\epsilon>0$. We then have that there exists $\delta>0$ such that for all $x \in [a,b]$, if $x \in (c-\delta, c)$,
$$|\frac{F(x)-F(c)}{x-c} - g(c)|$$
$$\leq \frac{1}{|x-c|} |\int_c^x |g(t) - g(c)| \, dt |$$
$$< \frac{1}{|x-c|} |\int_c^x \epsilon \, dt |$$
$$= \frac{1}{|x-c|} |x-c|\epsilon$$
$$= \epsilon$$
So the left derivative of $F$ at $c$ is $g(c)= \lim_\limits{x \to c^-} f(x)$.
The proof for the right derivative of $F$ at $c$ is similar, changing the obvious parts.
$\blacksquare$
But since $f$ has a jump discontinuity (which is a stronger condition that just being discontinuous at $c$, since some discontinuities are removable), $\lim_\limits{x \to c^-} f(x) \neq \lim_\limits{x \to c^+} f(x)$. So the left and right derivatives of $F$ at $c$ are not equal and hence $F$ is not differentiable at $c$.
Best Answer
One way to see that $f$ doesn't have an antiderivative, is to use Darboux's theorem which states that every derivative has the intermediate value property, even if it (the derivative) is not continuous.