Using Lagrange's mean value theorem. We have
$$\frac{f(x_0+\Delta x)-f(x_0)}{\Delta x}=f'(x_0+\theta \Delta x)$$
Taking limits on both sides, we see that the limit of the derivative always equals to the value of the derivative at $x_0$.
But here is a counter example:
$f(x)=x^2 \sin \frac 1 x$ , for $x\neq 0$ , And
$f(x)=0$ , for $x=0$
This function is continuous ,say, on the interval [-1,1]. Also, each point in it has a finite derivative. That is to say, it satisfies the condition of the mean value theorem.
But $f'(0)=0$, while $\lim_{x\to 0} f'(x)$ doesn't exist at all.
Can you explain this contradiction ?
Actually this's an example in Фихтенгольц's(Fikhtengolz) Calculus book. But he didn't clarify it.
Best Answer
The MVT depends on the range of appropriate values of $f'$ being present on the interval $(0,h)$, which is not the same thing as the limit. If you graph $f'$ you'll see it is highly oscillatory but $f'((0,h))$ should basically give you the same interval for all $h$ small.
Let me put it another way: Can you explain to me how the MVT implies continuity of $f'$? This is what you are assuming.
Here's an explanation for why you cannot take the limit of each side to conclude that $f'(0)=\lim_{h \to 0} f'(h)$. Consider the sequential formulation of limits, that is $\lim_{h \to 0} g(h)=L$ if for all $h_n\to 0 $ we have $g(h_n) \to 0$.
Let $h_n \to 0$ be arbitrary. The MVT states that for each $n$, we have $$\frac{f(h_n)-f(0)}{h_n}=f'(0+\theta(h_n)h_n).$$ Please note that $\theta(h_n)$ is in $(0,1)$ and depends on $h_n$. Taking $n \to \infty$ gives $$f'(0)=\lim_{n \to \infty} f'(\theta(h_n)h_n).$$ But, $\theta(h_n) h_n$ does not necessarily range through all sequences going to zero no matter how $h_n$ is picked. So we can't conclude that the RHS is $\lim_{h \to 0} f'(h)$.