I know the Taylor series expansion in single variable case:
$$ f(x) = f(x_0) + f'(x_0)(x – x_0) + \frac{1}{2}f''(x_0)(x-x_0)^2 + \frac{1}{3!}f^{(3)}(x_0)(x-x_0)^3 + \frac{1}{4!}f^{(4)}(x_0)(x-x_0)^4 + \frac{1}{5!}f^{(5)}(x_0)(x-x_0)^5 + \dots $$
Multivariable case is stated as:
$$ f(\bar{x}) = f(\bar{x_0}) + \bar\nabla f(\bar{x_0})^T(\bar{x} – \bar{x_0}) + \frac{1}{2}(\bar{x} – \bar{x_0})^T\bar\nabla^2 f(\bar{x_0})(\bar{x} – \bar{x_0}) + H.O.T. $$
I can find the expression above everywhere, but it is not possible to find the open expression for the H.O.T. (at least I wasn't able to find it). No source states the explicit expression for the higher order terms. Do people avoid them for some reason? For example, are those terms too complex or too long to write?
So, what are these higher order terms in the multivariable case? Do they involve a $\bar\nabla^3 f(\bar{x_0})$, $\bar\nabla^4 f(\bar{x_0})$, $\bar\nabla^5 f(\bar{x_0})$, … terms; if yes, how are they defined? Please write a few terms from H.O.T. to make the pattern clear.
Best Answer
The formula is almost identical to the single variable case. You should be able to find this in any good book covering calculus on Banach spaces.
Theorem. Let $E,F$ be Banach spaces, let $A\subseteq E$ be an open set and let $f:A\to F$ be a class $C^p$ map. Let $x\in A$ and let $v\in E$. Assume that the line segment $x+tv$ with $0\le t\le1$ is contained in $A$. Write $v^{(k)}$ for the $k$-tuple $(v,\dots,v)$. Then $$ f(x+v)=\sum_{k=0}^{p-1}\frac{f^{(k)}(x)v^{(k)}}{k!} + R_p, $$ where $$ R_p = \int_0^1 \frac{(1-t)^{p-1}}{(p-1)!} f^{(p)}(x+tv)v^{(p)}\,dt. $$
Proof. Integration by parts.
Here $f^{(k)}(x)\in L(E,L(E,\dots,L(E,F))\dots)$. (You should already know that the first derivative $f'(x)$ is just a linear map from $E$ to $F$, so we can differentiate the map $f':A\to L(E,F)$ to get $f'':A\to L(E,L(E,F))$ and so on.)
Check out Mathematical Analysis II by Zorich & Cooke or Real and Functional Analysis by Lang.