Could someone explain the justification and limits of fitting polynomials to arbitrary data points? I mean what about square roots or fractional or inverse powers?
Most of the time some wants to improve a linear fit, they rather include quadric terms than anything else.
Can you think of a mathematical justification why plain powers are favoured?
I assume it might be related to function approximation Taylor series.
Suppose I wanted to include fractional powers, would it rather make sense to include $x^a$ or $x^{0.5}+a\cdot x^{1.5}$ for "common real world data".
Or do rational polynomials do better and I'd rather try these first?
Maybe someone can elaborate the power of these method to approximate an unknown function in data.
Best Answer
There are lots of theoretical results telling us that approximation by polynomials works well for various classes of functions, and even telling us what the maximum approximation error will be. For example, there's the Stone-Weiertrass theorem mentioned in the other answer, plus the "Jackson" theorems and many others in constructive approximation: http://en.wikipedia.org/wiki/Constructive_function_theory
There are fast reliable easy-to-implement algorithms for computing the approximations. For some good examples, look at the Chebfun system, which basically does everything by computing high-degree polynomial approximations: http://www2.maths.ox.ac.uk/chebfun/
Once you have a polynomial, it's relatively easy (and inexpensive) to calculate function values, derivatives, integrals, zeros, bounds, and so on. Again, see Chebfun for examples.
In some fields (like computer-aided design), polynomial forms are considered "standard", and using anything else causes data exchange problems.
Rational approximations will sometimes work better than polynomial ones (in the sense that you get a smaller error with no increase in the degrees of freedom of the approximant). But optimal rational approximations are much harder to compute, and, once you have them, they are more difficult to handle (harder to integrate, for example).
Polynomial approximation is not always the best choice (nothing is), but it's often a pretty good one, and it's a reasonable thing to try unless the nature of your specific problem suggests something different.