It's easy to find a function which is continuous but not differentiable at a single point, e.g. $f(x) = |x|$ is continuous but not differentiable at $0$.
Moreover, there are functions which are continuous but nowhere differentiable, such as the Weierstrass function.
On the other hand, continuity follows from differentiability, so there are no differentiable functions which aren't also continuous. If a function is differentiable at $x$, then the limit $(f(x+h)-f(x))/h$ must exist (and be finite) as $h$ tends to 0, which means $f(x+h)$ must tend to $f(x)$ as $h$ tends to 0, which means $f$ is continuous at $x$.
I will assume that $a<b$.
Consider the function $g: [a,b]\to {\mathbb R}$ which equals $0$ at $a$, and equals $1$ on the interval $(a,b]$. This function is differentiable on $(a,b)$ but is not continuous on $[a,b]$. Thus, "we can safely say..." is plain wrong.
However, one can define derivatives of an arbitrary function $f: [a,b]\to {\mathbb R}$ at the points $a$ and $b$ as $1$-sided limits:
$$
f'(a):= \lim_{x\to a+} \frac{f(x)-f(a)}{x-a},
$$
$$
f'(b):= \lim_{x\to b-} \frac{f(x)-f(b)}{x-b}.
$$
If these limits exist (as real numbers), then this function is called differentiable at the points $a, b$. For the points of $(a,b)$ the derivative is defined as usual, of course. The function $f$ is said to be differentiable on $[a,b]$ if its derivative exists at every point of $[a,b]$.
Now, the theorem is that a function differentiable on $[a,b]$ is also continuous on $[a,b]$. As for the proof, you can avoid $\epsilon$-$\delta$ definitions and just use limit theorems. For instance, to check continuity at $a$, use:
$$
\lim_{x\to a+} (f(x)-f(a))= \lim_{x\to a+} (x-a) \lim_{x\to a+} \frac{f(x)-f(a)}{x-a} = 0\cdot f'(a)=0.
$$
Hence,
$$
\lim_{x\to a+} f(x)=f(a),
$$
hence, $f$ is continuous at $a$. For other points the proof is the same.
Best Answer
No. Weierstraß gave in 1872 the first published example of a continuous function that's nowhere differentiable.