[Math] Taylor’s theorem for vector valued functions

calculuslinear programmingnonlinear optimizationtaylor expansion

I'm reading about linear and nonlinear programming and on one page I have the following statment (I have highlighted the areas where I have problems and drawn questions for them in the bottom of it):

Proposition $5$. Let $f\in C^2$. Then $f$ is convex over a convex set $\Omega$ containing an interior point if and only if the Hessian matrix $\textbf{F}$ of $f$ is positive semidefinite throughout $\Omega$.

Proof. By Taylor's theorem we have
$$\color{#48BE6B}{\boxed{\,\,\,\displaystyle\color{black}{f(\mathbf{y})=f(\mathbf{x})\color{#ED1C24}{\boxed{\color{black}{=}}}\boldsymbol\nabla f(\mathbf{x})(\mathbf{y}-\mathbf{x})+\frac12(\mathbf{y}-\mathbf{x})^T\mathbf{F}\color{#FF7F27}{\boxed{\color{black}{(\mathbf{x}+\alpha(\mathbf{y}-\mathbf{x}))}}}(\mathbf{y}-\mathbf{x})}\,\,\,}}\tag{12}$$
for some $\alpha, 0\leqslant\alpha\leqslant1$. Clearly, if the Hessian is everywhere positive semidefinite, we have
$$f(\mathbf{y})\geqslant f(\mathbf{x})+\boldsymbol{\nabla}f(\mathbf{x})(\mathbf{y}-\mathbf{x}).\tag{13}$$
which in view of Proposition $4$ implies $f$ is convex.

Now suppose the Hessian is not positive semidefinite at some point $\mathbf{x}\in\Omega$. By continuity of the Hessian it can be assumed, without loss of generality, that $\mathbf{x}$ is an interior point of $\Omega$. There is a $\mathbf{y}\in\Omega$ such that $(\mathbf{y}-\mathbf{x})^T\mathbf{F}(\mathbf{x})(\mathbf{y}-\mathbf{x})<0$. Again by the continuity of the Hessian, $\mathbf{y}$ may be selected so that for all $\alpha, \,0\leqslant\alpha\leqslant1$.
$$(\mathbf{y}-\mathbf{x})^T\mathbf{F}(\mathbf{x}+\alpha(\mathbf{y}-\mathbf{x}))(\mathbf{y}-\mathbf{x})<0.$$
This in view of $\text{(12)}$ implies that $\text{(13)}$ does not hold; which in view of Proposition $4$ implies that $f$ is not convex. $\blacksquare$

$\begin{align}\color{#ED1C24}{\blacksquare}\,\,\,& \text{Is this a mistake? Should there be a + instead of =?}\\
\color{#FF7F27}{\blacksquare}\,\,\,& \text{I just don't understand this… Why is the alpha there?}\\
\color{#48BE6B}{\blacksquare}\,\,\,& \text{In general what is the Taylor theorem for vector valued}\\
& \text{functions? The formula in Wikipedia looks different than}\\ &\text{the formula here.}\\
\end{align}$

reference on wikipedia:

http://en.wikipedia.org/wiki/Taylor%27s_theorem#Taylor.27s_theorem_for_multivariate_functions

Any help much appreciated `=)

Best Answer

  • Yes there should be a plus.
  • You see this Taylor expansion uses "=" which means exactly the same. We call the orange part Lagrangian form of the remainder, which also can be found on the same page of wikipedia.
  • The general Taylor expansion is exactly what wiki writes. And the theorem in this book, the author takes the first order approximation, which is the simplest case of Taylor expansion. You can also expand the function to higher order according to the extend how precise is the approximation.
Related Question