To show $F$ is continuous at $c$:
We know that $f$ is Riemann-integrable because $f$ has only one discontinuity on $[a,b]$, and that discontinuity is a jump discontinuity (the one-sided limits of $f$ as $f$ approaches $c$ are finite), so $f$ is bounded on $[a,b]$. So by Lebesgue's criterion of Riemann integrability, $f$ is Riemann integrable on $[a,b]$.
Now $$F(x)-F(c)=\int_a^xf(t) \, dt-\int_a^cf(t) \, dt = \int_c^x f(t)\, dt$$
Since $f$ is bounded, for all $x \in [a,b]$, $|f(x)|\leq M$, for some $M \in \mathbb{R}$. Therefore: $$|\int_c^xf(t) \, dt| \leq \int_c^x|f(t)| \, dt \leq \int_c^xM \, dt$$
$$\implies -\int_c^x M\;dt\leq \int_c^x f(t)dt\leq \int_c^xM \;dt$$
$$\implies -M(x-c)\leq \int_c^x f(t)dt\leq M(x-c)$$
Taking the limit as $x$ tends to $ c$, the squeeze theorem says $\lim\limits_{x\to c}F(x)=F(c)$. So $F$ is continuous at $c$.
$\blacksquare$
To show that $F$ is not differentiable at $c$:
With help from Showing that an indefinite integral of a function with a jump discontinuity is not differentiable at the jump discontinuity
Lemma: The left derivative of $F(x)$ at $c$ is $\lim_\limits{x\to c^-} f(x)$ and the right derivative of $F(x)$ at $c$ is $\lim_\limits{x\to c^+} f(x)$.
Proof: Define $g:[a,c]\to \mathbb{R} $ by:
$$g(x) = \begin{cases}
f(x) & \text{if } x<c \\ \lim_\limits{x \to c^-} f(x) & \text{if } x = c
\end{cases}
$$
Then $g$ is continuous on $[a,c]$.
Now for all $x \in [a,c]$, $F(x) = \int_a^x f(t) \,dt =\int_a^x g(t) \,dt $. Why? Well if we interpret the Riemann integral as a Lebesgue integral, this is very easy since $f=g$ almost everywhere on $[a,c]$. If you want all the technical details with Riemann integrals see here: Proving Riemann integral does not change when finite values of a function is changed.
Now we show that the continuity of $g$ implies that the left derivative of $F$ at $c$ is $g(c)$.
We note that $g(c)$ is a constant, so $(x-c)g(c) = \int_c^x g(c) dt$, hence $g(c) = \frac{\int_c^x g(c) dt}{x-c}$.
Then given $x \neq c$,
$$|\frac{F(x)-F(c)}{x-c} - g(c)|$$
$$= |\frac{\int_a^x g(t) \, dt - \int_a^c g(t) \, dt}{x-c} - \frac{\int_c^x g(c) dt}{x-c}| $$
$$= |\frac{\int_c^x g(t) \, dt}{x-c} - \frac{\int_c^x g(c) dt}{x-c}| $$
$$= \frac{1}{|x-c|} |\int_c^x (g(t) - g(c)) \, dt |$$
$$\leq \frac{1}{|x-c|} |\int_c^x |g(t) - g(c)| \, dt |$$
Since $g$ is continuous at $c$, there exists a $\delta>0$ such that for all $x \in [a,c]$, if $|x-c|<\delta$, then $|g(x)-g(c)|<\epsilon$.
So take any $\epsilon>0$. We then have that there exists $\delta>0$ such that for all $x \in [a,b]$, if $x \in (c-\delta, c)$,
$$|\frac{F(x)-F(c)}{x-c} - g(c)|$$
$$\leq \frac{1}{|x-c|} |\int_c^x |g(t) - g(c)| \, dt |$$
$$< \frac{1}{|x-c|} |\int_c^x \epsilon \, dt |$$
$$= \frac{1}{|x-c|} |x-c|\epsilon$$
$$= \epsilon$$
So the left derivative of $F$ at $c$ is $g(c)= \lim_\limits{x \to c^-} f(x)$.
The proof for the right derivative of $F$ at $c$ is similar, changing the obvious parts.
$\blacksquare$
But since $f$ has a jump discontinuity (which is a stronger condition that just being discontinuous at $c$, since some discontinuities are removable), $\lim_\limits{x \to c^-} f(x) \neq \lim_\limits{x \to c^+} f(x)$. So the left and right derivatives of $F$ at $c$ are not equal and hence $F$ is not differentiable at $c$.
A function $f$ cannot be both unbounded and Riemann-Stieltjes integrable.
This can be shown by producing an $\epsilon > 0$ such that for any real number $A$ and any $\delta > 0$ there is a tagged partition $P$ with $\|P\| < \delta$ and with a Riemann-Stieltjes sum satisfying
$$|S(P,f,\alpha) - A| > \epsilon$$
Given any partition $P$, since $f$ is unbounded, it must be unbounded on at least one subinterval $[x_{j-1},x_j]$ of P. Using the reverse triangle inequality we have
$$|S(P,f,\alpha) - A| = \left|f(t_j)(\alpha(x_j) - \alpha(x_{j-1})) + \sum_{k \neq j}f(t_k)(\alpha(x_k) - \alpha(x_{k-1})) - A \right| \\ \geqslant |f(t_j)|(\alpha(x_j) - \alpha(x_{j-1})) - \left|\sum_{k \neq j}f(t_k)(\alpha(x_k) - \alpha(x_{k-1})) - A \right|$$
Since $f$ is unbounded on $[x_{j-1},x_j]$, choose a partition tag $t_j$ such that
$$|f(t_j)| > \frac{\epsilon + \left|\sum_{k \neq j}f(t_k)(\alpha(x_k) - \alpha(x_{k-1})) - A \right|}{\alpha(x_j) - \alpha(x_{j-1})},$$
and it follows that no matter how fine the partition $P$ we have
$$|S(P,f, \alpha) - A| > \epsilon.$$
Thus, when $f$ is unbounded, it is impossible to find $A$ such that for every $\epsilon > 0$ and sufficiently fine partitions, the condition $|S(P,f,\alpha) - A| < \epsilon$ holds. We can always select the tags so that the inequality is violated.
Best Answer
Edit: I understood integrability in the Lebesgue sense. The longer first part is the original answer.
I'm rearranging my answer since I'm a bit confused about the actual question, which seems to be:
The answer to that question is no, since we can always modify $f$ on a null set without changing $F$. The characteristic function of $\mathbb{Q}$, for example, is nowhere continuous but its integral is zero over every interval. These are true discontinuities, not only removable ones.
The point is that $f$ is completely underdetermined in this question. Any modification of $f$ on a null set will yield the same $F$, thus we can't hope for more than $F' = f$ almost everywhere. Amazingly enough, this turns out to be true.
The right question to ask was answered by Lebesgue. The starting point is that for an integrable function $f$ its definite integral $F$ will always be absolutely continuous. The Lebesgue differentiation theorem even tells us that $F$ is almost everywhere differentiable and $F' = f$ almost everywhere.
Lebesgue proved even more:
This is proved in any decent text on measure theory e.g. Royden or Rudin.
On the other hand, Luzin's theorem implies that a measurable $f$ is continuous on the complement on a set of arbitrarily small measure.
One further point you should be aware of: If $F$ is continuous and differentiable almost everywhere and $F' = 0$ wherever it is defined then $F$ can be non-constant, even monotonic. the standard example for this is called the Cantor-Lebesgue function, or more colorfully the devil's staircase.
Finally, I'd like to point out that it is an awfully subtle business to characterize the functions which are derivatives of everywhere differentiable functions.
Added:
Robert Israel left the following comment to this answer (thank you very much for that!):
The only thing I'd like to add to that is:
see Über die Darstellbarkeit einer Funktion durch eine trigonometrische Reihe (1854). More precisely, the relevant passage is Section 5 on pages 226 and 227 of Bernhard Riemann's Gesammelte mathematische Werke und wissenschaftlicher Nachlass.
(Hat tip to Roy Smith and Pete L. Clark)