Here is the list of references I've seen on the topic of elementary integration. It is eclectic, and not intended to be complete, but contains most of what seem to be the relevant benchmarks, and several expository accounts in varying degrees of detail. Sadly, I am not familiar with Liouville's or Ostrowski's original papers. (Perhaps I'll use this as an excuse to track them down.)
MR0223346 (36 #6394). Rosenlicht, Maxwell. Liouville's theorem on functions with elementary integrals. Pacific J. Math. 24 (1968), 153–161.
MR0237477 (38 #5759). Risch, Robert H. The problem of integration in finite terms. Trans. Amer. Math. Soc. 139 (1969), 167–189.
MR0269635 (42 #4530). Risch, Robert H. The solution of the problem of integration in finite terms. Bull. Amer. Math. Soc. 76, (1970), 605–608.
MR0321914 (48 #279). Rosenlicht, Maxwell. Integration in finite terms. Amer. Math. Monthly 79 (1972), 963–972.
MR0409427 (53 #13182). Risch, Robert H. Implicitly elementary integrals. Proc. Amer. Math. Soc. 57 (1), (1976), 1–7.
MR0536040 (81b:12029). Risch, Robert H. Algebraic properties of the elementary functions of analysis. Amer. J. Math. 101 (4), (1979), 743–759.
MR0815235 (87a:12009). Richtmyer, R. D. Integration in finite terms: a method for determining regular fields for the Risch algorithm. Lett. Math. Phys. 10 (2-3), (1985), 135–141.
Matthew P Wiener. Elementary integration and $x^x$. Sci.Math post. February 21, 1995. (The pdf version was typed by Apollo Hogan).
Manuel Bronstein. Symbolic Integration Tutorial. "Course notes of an ISSAC (International Symposium on Symbolic and Algebraic Computation) '98 tutorial."
MR1960772 (2004c:12010). Van der Put, Marius; Singer, Michael F. Galois theory of linear differential equations. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], 328. Springer-Verlag, Berlin, 2003. xviii+438 pp. ISBN: 3-540-44228-6.
MR2106657 (2005i:68092). Bronstein, Manuel. Symbolic integration. I.
Transcendental functions. Second edition. With a foreword by B. F. Caviness. Algorithms and Computation in Mathematics, 1. Springer-Verlag, Berlin, 2005. xvi+325 pp. ISBN: 3-540-21493-3.
Brian Conrad. Integration in elementary terms. Unpublished note. (2011?).
Moshe Kamensky. Differential Galois theory. "An introduction to Galois theory of linear differential equations."
Bronstein's book in particular is highly recommended.
There are an infinite number of functions out there, and you can put an integral sign in front of any of them. Some of those functions are pretty strange and/or ugly. There is no reason why such an integral should have a representation in the very limited set of elementary functions. In fact when you get such a representation, you could consider yourself lucky.
For practical purposes, any convergent integral can be evaluated to any desired degree of accuracy. There are numerous numeric methods available with which to do that. And even if an integral should have some elementary function representation, that most often is a lot harder to evaluate than just running a numeric method on the integral. And you wouldn't be more accurate if the answer were cos(28.34) and you had to evaluate that.
Similarly, if you are going to use the integral in a further computation, it might be easiest to leave it as an integral, rather than tangling yourself up in things that have a lot of terms like sec(log^{-1}(x^{2})) -- or much worse.
So why do calculus classes teach you to find anti-derivatives? First, if the problem is going to resolve itself into an easy anti-derivative you might as well use it. Second, to familiarize you better with the underlying concepts, such as the chain rule and the product rule of derivatives. Third to give you some experience with actual answers so you have some idea what the answer ought to look like. If your numeric method evaluates to 2034.86, and you know the answer can't be larger than 80, then you know you made a mistake in your computation.
Of course, nowadays, all those integrals can be accurately evaluated online; but still, you should have some knowledge of what kind of answer to expect and what it all means.
Best Answer
Two reasons:
1) Not all integrable functions are differentiable, not all differentiable functions are smooth (i.e., have derivatives of all orders, which is necessary for the series to exist), and not all smooth functions converge to their Taylor series. The canonical example for the latter is $f(x) = e^{-1/x^2}$ (with $f(0) = 0$), which is smooth with $f^{(n)}(0) = 0$ for all $n$ and thus has a Taylor series around $0$ that's exactly $0$.
2) An arbitrary Taylor series is not a polynomial or even an elementary function. Take $\operatorname{erf}(x) = \frac{2}{\sqrt{\pi}}\int_0^x dt\; e^{-t^2}$, for example, or $\Gamma(x) = \int_0^\infty dt\;t^{x-1}e^{-t}$.