My textbook has written the following for the second-order Taylor formula of Taylor's theorem for many variables:
$$f(\mathbf{x}_0 + \mathbf{h}) = f(\mathbf{x_0}) + \sum_{i = 1}^n h_i \dfrac{\partial{f}}{\partial{x_i}}(\mathbf{x}_0) + \dfrac{1}{2} \sum_{i, j = 1}^n h_i h_j \dfrac{\partial^2 f}{\partial{x_i} \partial{x_j}} (\mathbf{x}_0) + R_2(\mathbf{x}_0, \mathbf{h}),$$
where
$$R_2(\mathbf{x}_0, \mathbf{h}) = \sum_{i, j, k = 1}^n \int_0^1 \dfrac{(t – 1)^2}{2} \dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}} (\mathbf{x_0 + t \mathbf{h}})h_i h_j h_k \ dt.$$
The integrand is a continuous function of $t$ and is therefore bounded by a positive constant $C$ on a small neighbourhood of $\mathbf{x}_0$ (because it has to be close to its value at $\mathbf{x}_0$). Also note that $|h_i| \le ||\mathbf{h}||$, for $||\mathbf{h}||$ small, and so
$$|R_2(\mathbf{x}_0, \mathbf{h})| \le ||\mathbf{h}||^3C$$
I have a number of points of confusion:
-
The integrand is a continuous function of $t$ and is therefore bounded by a positive constant $C$ on a small neighbourhood of $\mathbf{x}_0$ (because it has to be close to its value at $\mathbf{x}_0$).
Can someone please explain this?
-
Also note that $|h_i| \le ||\mathbf{h}||$, for $||\mathbf{h}||$ small, …
Why is this the case? And why only for $||\mathbf{h}||$ small?
-
$$|R_2(\mathbf{x}_0, \mathbf{h})| \le ||\mathbf{h}||^3C$$
I'm trying to figure out where this came from. On a previous page, the textbook states the following:
$$| R_k(x_0, h) | = \left| \int_{x_0}^{x_0 + h} \dfrac{(x_0 + h – \tau)^k}{k!} f^{k + 1}(\tau) \ d \tau \right| \le \dfrac{|h|^{k + 1}}{k!} M$$
But, if we have $||\mathbf{h}||^3C$, then, according to the above statement, we would need $k = 2$; but the $2!$ factor is missing, so it seems that this assumption might be incorrect? So where did this come from?
I would greatly it if people could please take the time to clarify these points of confusion.
Best Answer
Since $\dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}}$ is continuous at the point $ \mathbf{x}_0$, if you take $\varepsilon=1$, using the definition of continuity you can find $\delta=\delta(\mathbf{x}_0,1)>0$ such that $$\left\vert \dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}}(\mathbf{x})-\dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}}(\mathbf{x}_0)\right\vert\le 1$$ for all $\mathbf{x}$ in the domain with $\Vert\mathbf{x}-\mathbf{x}_0\Vert\le\delta$. Hence, by the triangle inequality you get \begin{align}\left\vert \dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}}(\mathbf{x})\right\vert&\le \left\vert\dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}}(\mathbf{x}_0)\right\vert\\&+\left\vert \dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}}(\mathbf{x})-\dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}}(\mathbf{x}_0)\right\vert\\&\le \left\vert\dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}}(\mathbf{x}_0)\right\vert+1:=M\end{align} for all $\mathbf{x}$ in the domain with $\Vert\mathbf{x}-\mathbf{x}_0\Vert\le\delta$. In particular, if you now take $\mathbf{x}=\mathbf{x}_0+t\mathbf{h}$ with $\Vert\mathbf{h}\Vert\le\delta$ you have that $$\left\vert \dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}}(\mathbf{x}_0+t\mathbf{h})\right\vert\le M$$ for all $t\in [0,1]$ and all $\Vert\mathbf{h}\Vert\le\delta$. Hence, the function inside the integral can be bound in absolute value by \begin{align}\left\vert\dfrac{(t - 1)^2}{2} \dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}} (\mathbf{x_0 + t \mathbf{h}})h_i h_j h_k\right\vert\\\le \dfrac{(t - 1)^2}{2}\left\vert \dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}}(\mathbf{x}_0+t\mathbf{h})\right\vert\vert h_i h_j h_k\vert \le\frac12M\Vert\mathbf{h}\Vert^3.\end{align} In turn, \begin{align}\vert R_2(\mathbf{x}_0, \mathbf{h})\vert &\le \sum_{i, j, k = 1}^n \int_0^1 \left\vert\dfrac{(t - 1)^2}{2} \dfrac{\partial^3{f}}{\partial{x_i} \partial{x_j} \partial{x_k}} (\mathbf{x_0 + t \mathbf{h}})h_i h_j h_k\right\vert \ dt\\&\le \frac12M\Vert\mathbf{h}\Vert^3 \sum_{i, j, k = 1}^n \int_0^1 1\, dt=\frac12M\Vert\mathbf{h}\Vert^3 \sum_{i, j, k = 1}^n1 \end{align} for all $\Vert\mathbf{h}\Vert\le\delta$. The inequality $\vert h_i\vert\le \Vert\mathbf{h}\Vert$ is always true since $$\Vert\mathbf{h}\Vert=\sqrt{\sum_{j=1}^nx_j^2}\ge \sqrt{x_i^2}=\vert x_i\vert. $$