We let $I=[a,b]$ and consider a partition $P=\{a=x_{0}<x_{1}<\ldots<x_{n}=b\}$ of $I$.
Next observe that for each $s,t\in[x_{i-1},x_{i}]$, where $i=1,2,\ldots,n$ and $s<t$, that
\begin{equation*}
\frac{F(t)-F(s)}{t-s}=\frac{1}{t-s}\int_{s}^{t}\!{}f(x)\mathrm{d}x
\ge\inf_{[x_{i-1},x_{i}]}(f).
\end{equation*}
Letting $s$ increase to $t$ gives that
\begin{equation*}
F'(t)\ge\inf_{[x_{i-1},x_{i}]}(f).
\end{equation*}
Since $t\in[x_{i-1},x_{i}]$ was arbitrary and $i=1,2,\ldots,n$ was arbitrary we have
\begin{equation*}
\inf_{[x_{i-1},x_{i}]}(F')\ge\inf_{[x_{i-1},x_{i}]}(f)
\end{equation*}
for all $i=1,2,\ldots,n$. Note that a similar proof can be used to establish this inequality at $t=x_{i-1}$. A similar proof also shows that
\begin{equation*}
\sup_{[x_{i-1},x_{i}]}(F')\le\sup_{[x_{i-1},x_{i}]}(f)
\end{equation*}
for all $i=1,2,\ldots,n$. We conclude that
\begin{equation*}
L(f,P)\le{}L(F',P)\le{}U(F',P)\le{}U(f,P)
\end{equation*}
where $L(g,P)$ and $U(g,P)$ denote the lower and upper Darboux sums of $g$ over $P$.
Refining the partition and using that $f$ is integrable shows that $F'$ is integrable over $I$.
Best Answer
The fundamental theorem of calculus says that $F$ is continuous, and differentiable at every point where $f$ is continuous. So to get an example, you must take $f$ discontinuous. For example, if $f(x)=0$ for $x<0$ and $f(x)=1$ for $x\ge0$, then (with $a=0$) we get $F(x)=0$ for $x<0$ and $F(x)=x$ for $x\ge0$, so $F$ is not differentiable at the point $x=0$.