[Math] Interpolation, Extrapolation and Approximations rigorously

approximationapproximation-theorydefinitionlagrange-interpolationterminology

A foreign book mentioned that

"when the Lagrange's interpolation formula fails (for example with large sample due to Runge's phenomenon), you should use approximation methods such as Least-squares-method."

I am confused because I have always thought that interpolation/extrapolations are approximations. My confusion lies in the fact that the book used the three terms as disjoint while I have considered the first two terms as approximating.

For example, can the Lagrange polynomial (also known as Lagrange interpolation) be extrapolation, interpolation and approximation at the same time? I would say yes and cannot see no problem to use the L polynomial to create extrapolations and approximations (I feel the terms fuzzy).

So how do you define the terms about approximation more rigorously?

Best Answer

"Approximation" is a very general term, whose definition I'll defer to a later paragraph. For the time being, I concentrate on interpolation and extrapolation.

The assumption here is that you are given a set of points $(x_k,y_k),\quad k=1\dots n$. Interpolation corresponds to finding a function $f(x)$ (e.g. a polynomial) such that $y_k=f(x_k)$ for all $k$.

Usually, the points $(x_k,y_k)$ are within some interval or region, e.g. $x_1 < x_2 < \dots < x_n$. Extrapolation is essentially evaluating the interpolating function $f(x)$ at a value of $x$ outside the region/interval containing the original points, e.g. some value $w < x_1$, or $z > x_n$. This can be slightly risky, since an interpolating function being well-behaved at a neighborhood of the interpolation points does not at all guarantee that the function will still behave nicely outside the original interval/region (and even then you see things like the Runge phenomenon, where the interpolant wiggles widely in between interpolation points).

Approximation, as I've said, is a very general term. One usually does not speak of interpolants as "approximants", since interpolants pass through each given point by construction. However, by a slight abuse of terminology, one sometimes considers the evaluation of some "difficult" function $f(x)$ by taking its values at known points, and constructing an interpolant from these known points. In this respect, the "interpolant" here is called an "approximant". (In my opinion, this (usually) isn't the best way to obtain an approximant!)

One might also consider the more general case of the given points having some form of error; here, one usually fits instead of interpolating. I talked about that in this answer, so I won't be repeating myself...