The way I teach it, $\int f(x)dx$ is a symbol which stands for the entire set of solutions to the differential equation $\frac{dy}{dx} = f(x)$. The content of the fundamental theorem is that to evaluate a definite integral on an interval, one need only evaluate the difference between the value of the end points for any member of this set.
I do agree that the notation is confusing, because it looks like it represents one function, when in fact it stands for a whole class of functions. Even writing $+C$ does not fix everything, since your domain might be disconnected. Well, it does fix everything if you interpret $C$ as a locally constant function...
No, not always.
In fact, the definite integral with variable bounds can be and is strictly more restrictive than the indefinite integral, in the most general case where both can be taken as sensibly existing to their fullest extent that we can say one defines the same kind of function as the other.
While it is true that if we have two functions $F_1$ and $F_2$ defined by definite integrals of the same function with different bounds, i.e.
$$F_1(x) := \int_{a_1}^{x} f(t)\ dt\ \mathrm{and}\ F_2(x) := \int_{a_2}^{x} f(t)\ dt$$
with $a_1 \ne a_2$, then we will have that $F_1 - F_2$ is a constant function, the range of possible such constants may be limited. A simple counterexample is to take $f(x) := \sin(x)$. Now the integral from any lower bound $a$ to $x$ is just $\int_a^x \sin(t)\ dt = \cos(a) - \cos(x)$, but here $\cos(a)$ - the "constant" of our integration - can only range in $[-1, 1]$ (and the difference of any two such "constants" only within $[-2, 2]$), but the constant in $\int \sin(x)\ dx = -\cos(x) + C$ can be any real number whatsoever, even one far outside this range, say assign $C := \mbox{Graham's number}$. Thus the integrals with variable bounds are not exhaustive of the full family of antiderivatives, which is what is represented by the indefinite integral, and if we are really pedantic should be truly written as the set it is: in "reality", not in the informalities of school calculus,
$$\int f(x)\ dx = \left\{ F(x) + C : C \in \mathbb{R} \right\}$$
where $F'(x) = f(x)$. Actually, if we want to be really correct - and in fact this is important but only in cases where the definite integral technically doesn't make sense as evaluable between arbitrary pairs of points - we should take
$$\int f(x)\ dx = \left\{ F(x) \in \mathbb{R}^\mathbb{R} : \mbox{$F'$ exists and $F'(x) = f(x)$}\ \right\}$$.
Best Answer
Not quite. The arbitrary constant from integration changes it up. Explicitly,
$$\frac{d}{dx} \int f(x)dx = f(x)$$
$$\int \left( \frac{d}{dx} f(x) \right) dx = f(x) + C$$
An example will be illustrative in finding the anomaly if it's not obvious. Let $f(x) = x^2$. We know $\int f(x)dx = x^3/3 + C$, $f'(x) = 2x$, and, obviously, the derivative of a constant $C$ is $0$.
Then
$$\frac{d}{dx} \int f(x)dx = \frac{d}{dx} \int x^2dx = \frac{d}{dx} \left( \frac{x^3}{3} + C \right) = \frac{3x^2}{3} + 0 = x^2 = f(x)$$
but
$$\int \left( \frac{d}{dx} f(x) \right) dx = \int \left( \frac{d}{dx} x^2 \right) dx = \int 2xdx = \frac{2x^2}{2} + C = x^2 + C = f(x) + C$$