Let $f:\mathbb{R}^m\to\mathbb{R}^n$ so that $f\in C^{k,1}$, i.e. $f$ is $k$ times continuously derivable, and $D^kf$ is Lipschitz-continuous.
I'm looking for a Taylor expansion with remainder which makes use of the fact that $D^k f$ is Lipschitz to improve upon the size of the remainder.
For example, according to the book "Linear and Nonlinear Optimization" by Griva for $f\in C^{1,1}$ we have:
$$
\|f(x+p) -f(x) – (Df)(x)p\| \le \frac12 L \|p \| ^2
$$
and can therefore write the Taylor-expansion of $f$ in $x$ in direction $p$ as
$$
f(x+p)= f(x) + (Df)(x)\cdot p + O(\|p\|^2)
$$
Can this result be transferred to give better approximations for the remainder of higher taylor-approximations?
Given that a Lipschitz function $g$ is almost everywhere differentiable, we still have $\int_a^b g'(x) dx = g(b)-g(a)$.
So for $f\in C^{k,1}$ we can use the integral remainder
$$
(k+1) \int\limits_0^1 \sum_{|\alpha| = k + 1} \frac{(1 – t)^k p^\alpha}{\alpha !} D^{\alpha}f(x + tp) \, \mathrm{d}t
$$
However, since $D^{\alpha}f$ isn't continuous, I'm not sure whether we can bound it against $O(\|p^{k+1}\|)$
Best Answer
The answer is yes and it is demonstrated using Taylor's polynomial as shown below. I assume you know vector calculus. Unfortunately, people don't learn vector calculus anymore and I do not know any modern reference to it, I am using Foundation of Modern Analysis by Jean Dieudonne, ch. 8.
Taylor's theorem with integral remainder gives $$ f(x + h) = \mathbf{T}_{k - 1} f(x) \cdot h + \int\limits_0^1 dt\ \dfrac{(1 - t)^{k-1}}{(k-1)!} \mathbf{D}^{k}f(x + th) \cdot h^{(k)}, $$ where $\mathbf{T}_{k-1}f(x) \cdot h$ is the Taylor polynomial of $f$ or order $k-1$ evaluated at $h,$ $\mathbf{D}^kf(x)$ is the canonical $k$-linear function associated with the $k$th derivative of $f$ at $x$ and $h^{(k)} = (h, \ldots, h)$ ($k$ times).
We add and subtract $\dfrac{1}{k!} \mathbf{D}^k f(x) \cdot h^{(k)},$ so that $$ f(x + h) = \mathbf{T}_k f(x) \cdot h + I $$ and you want to show that $\|I\| \leq c \|h\|^{k+1}.$
First notice that $$ \dfrac{1}{k!} \mathbf{D}^k f(x) \cdot h^{(k)} = \int\limits_0^1 dt\ \dfrac{(1 - t)^{k - 1}}{(k - 1)!} \mathbf{D}^k f(x) \cdot h^{(k)}. $$ Second, if the Riemann integral of a vector valued function exists, as well as that of its norm, then we have the triangle inequality (since we are in finite dimensional spaces and everything is continuous, existence is not an issue here) $$ \left\| \int\limits_a^b dt\ \varphi(t) \right\| \leq \int\limits_a^b dt\ \|\varphi(t)\|. $$ Third, if $M$ is a $k$-linear function, then the fundamental inequality for the norm is $$ \| M(v_1, \ldots, v_k) \| \leq \|M\| \|v_1\| \cdots \|v_k\|. $$ Using these three properties, we can easily see $$ \| I \| \leq \left( \int\limits_0^1 dt\ \dfrac{(1 - t)^{k - 1}}{(k-1)!} \Big\| \mathbf{D}^k f(x + th) - \mathbf{D}^k f(x) \Big\| \right) \left\| h \right\|^k. $$ Since the $k$th derivative is Lipschitz continuous, the norm inside the integral above is bounded from above by $c \|t h\| \leq c \|h\|$ since $0 \leq t \leq 1.$ This finally yields $\| I \| \leq c' \|h\|^{k+1}. \square$
Scholium. All the properties above can be extended to normed spaces with appropriate restrictions, usually one assumes they are complete (a.k.a. Banach spaces) but even that is not necessary. However, when the codomain of $f$ is not complete, it is difficult to prove the existence of the Riemann integral, yet, the properties of the Riemann integral continue to hold provided said integral exists.