Given a function $f: \mathbb {R} ^n \to \mathbb {R} $, I am curious about the uniqueness of a $k$th-order approximation at $c \in \mathbb {R}^n $, i.e. a function $\phi(x)$ such that
$$ \frac {f(c + h) – \phi(c + h)} {\| h \|^k } \overset{h\to 0} { \to } 0,$$
and I want to show that there is a unique such $\phi$ that is a polynomial of degree $\leq k$ in $(x_1 – c_1, \dotsc , x_n – c_n) = (h_1 , \dotsc, h_n)$ (supposing that one exists at all). Of course, this will be the Taylor expansion of order $k$ at $c$. We can prove the uniqueness of the first order as follows: if $p_1$, $p_1'$ are polynomials of degree $\leq 1$ in the components, then they are of the form $$p_1 = v + \langle w, h \rangle \\ p_1' = v' + \langle w', h \rangle,$$
so since $\frac {f- p_1} {\|h\|}$ and $ \frac {f – p_2} { \| h \| } \to 0$ as $h\to 0$, we have that
$$ \frac { p_2 – p_1 } { \| h \| } = \frac {v-v'} { \| h \| } +\frac { \langle w-w' , h \rangle } { \|h\| }, $$
and if we let $h \to 0$ parallel to $w-w'$, we see that this can go to zero iff $v = v', w = w'$.
If we try to do the same thing in second order, we'll end up with
$$ \frac {v – v'} {\| h \|^2 } + \frac { \langle w – w', h \rangle} {\| h\|^2} + \frac {h^T (H – H')h} {\|h \|^2} $$ goes to zero, and we'd like to conclude from this that $H = H'$ etc. I'm not exactly sure how we'd do this; we'd maybe like to use the matrix norm, but that gives an upper bound, and we need a lower bound (something to force to zero).
Anyway, my questions are:
-
How can we show the uniqueness of such a $k$th order approximation in general?
-
Given a $k$th order approximation which is also a polynomial of degree $\leq k$ in the $h_i$, how can we show that those terms which are of degree $\leq k-1$ form a $k-1$th-order approximation?
It would be a bonus if we can answer questions 1 and 2 without reference to the derivative, and then afterward we can of course show that the existence of such an approximation is equivalent to $k$-times differentiability at $c$ (sometimes it is even taken as the definition).
Edit: @40votes has pointed out that we cannot reason from the existence of such an approximation to the existence of a derivative of order higher than 1.
Best Answer
If two polynomials $p,q$ of degree at most $k$ satisfy $$\lim_{h\to 0}\frac{p(c+h)-q(c+h)}{\|h\|^k}=0\tag1$$ for some $c$, then $p\equiv q$. Indeed, the difference $p-q$, being a polynomial of degree at most $k$, admits a finite Taylor expansion $$p(c+h)-q(c+h) = \sum_{|j|\le k} c_j h^j\tag2$$ where $j$ is a multiindex. Using (1) repeatedly with $k$ replaced by $0,1,2,\dots,k$ we find that $c_j=0$ for all $j$.
Given $p(c+h)=f(c+h)+o(\|h\|^k)$, write $p=q+p_k$ where $p_k$ contains the monomials in $h$ that are of degree precisely $k$, and $q$ has degree $<k$. Note that $p_{k}(c+h)=O(\|h\|^k)$ which is $o(\|h\|^{k-1})$. Hence $$p(c+h)-p_{k}(h)=f(c+h)+o(\|h\|^{k-1})$$
The existence of $k$th order approximation, for any $k\ge 1$, implies first-order differentiability. However, it does not imply the existence of second or higher order derivatives. A standard counterexample is $f(x)=x^{k+1}\sin x^{-k-1}$. The problem is that a bound on $f$ gives no control over the oscillation of $f'$. A more extreme example of this kind is $$f(x)=\exp (-x^{-2} ) \chi_{\mathbb Q}$$ which is discontinuous everywhere except $0$ but has polynomial approximation of all orders at $0$. (In both examples above, the function is extended to $x=0$ by $0$.)
In convex analysis and some areas of PDE one encounters pointwise second-order derivative, e.g., Second order differentiability of convex functions in Banach spaces by Borwein and Noll. These precisely correspond to the existence of 2nd order approximation described in the question. One can similarly define the pointwise $k$th derivative of $f$ as the corresponding derivative of its $k$th order polynomial approximation. I haven't seen this used for $k\ge 3$, although the concept is natural.
A related concept is jet.