Let $X$ be some Banach space, if $f \in C(U,\Bbb{R})$ a continuous real value function.We define the Gateaux derivative exist if for any $h\in X$ exist $df(x_0,h) \in \Bbb{R}^1$ s.t:
$$|f(x_0+th)-f(x_0)-tdf(x_0,h)|= o(t)$$
For Frechet derivative $f'(x_0)$ exist if for some $\xi\in X^*$:
$$|f(x)-f(x_0)-\langle\xi,x-x_0\rangle| = o(\|x-x_0\|)$$
Denote this $\xi = f'(x_0)$ the Frechet derivative
Prove the following result:
If $f$ exist Gateaux derivative everywhere inside a neiborhood $V$ around $x_0$ ,for each $x\in V$ exist some $\xi(x)\in X^*$ such that $$\langle \xi(x),h\rangle = df(x,h)$$
Moreover $x\mapsto \xi(x)$ is continuous then $f$ also has Frechet derivative at $x_0$.
To check Frechet derivative exist at the point $x_0$,needs to find some $\xi$ satisfies the definition,it's natural to guess that $\xi(x_0)\in X^*$ it's the desired one.
Choose $h = (x-x_0)/\|x-x_0\|$ then $\langle \xi(x_0),h\rangle = df(x_0,h)
$
Substitute into $$|f(x)-f(x_0)-\langle\xi(x_0),x-x_0\rangle| =\\ |f(x_0 +(x-x_0))-f(x_0)-\|x-x_0\|\langle\xi(x_0),h\rangle|
\\=|f(x + \|x-x_0\|h)-f(x_0)- \|x-x_0\|df(x_0,h)|$$
As $\|x-x_0\|\to 0$,it goes to zero,but this gives a wrong proof,the reason is we take $x-x_0$ direction fixed ,and the convergence is not uniformly over all the directions.
We need to use continuity of $\xi(x)$ to handle this problem.By definition we have for unit length $h$ ,$$|df(h,x)-df(h,x_0)| = |\langle\xi(x)-\xi(x_0),h\rangle|\le \|\xi(x)-\xi(x_0)\| \to 0$$ when $\|x-x_0\|\to 0$ I have no idea how to use this condition.This condition says "directional derivative" is continuous in all the directions.
My idea to proceed the proof is using the same trick in Rudin's mathematical analysis book p219 9.21:that construct a sequence of line segments.
$$f(\mathbf{x}+\mathbf{h})-f(\mathbf{x})=\sum_{j=1}^{n}\left[f\left(\mathbf{x}+\mathbf{v}_{j}\right)-f\left(\mathbf{x}+\mathbf{v}_{j-1}\right)\right]$$
with $\mathbf{v}_{k}=h_{1} \mathbf{e}_{1}+\cdots+h_{k} \mathbf{e}_{k}$ the problem here is we lie in infinite dimension space,this trick seems not apply well in our case?
Best Answer
Choose $\epsilon>0$, then note
$$ \begin{split} f(x+&h)-f(x)-df(x,h) \\ &=f(x+(1-t)h+th)-f(x+(1-t)h)\\ & \quad+f(x+(1-t)h) -f(x)-tdf(x,h)-(1-t)df(x,h)\\ &=tdf(x+(1-t)h,h)+o(|t|)+(1-t)df(x,h) \\ & \quad +o(|1-t|)-tdf(x,h)-(1-t)df(x,h)\\ &=t(df(x+(1-t)h,h)-df(x,h))+o(|t|)+o(|1-t|) \end{split} $$ So $$|f(x+h)-f(x)-df(x,h)|\leq|t||df(x+(1-t)h,h)-df(x,h)|+|o(|t|)|+|o(|1-t|)|$$ as $t$ is arbitrary, we can choose a separate $t$ for any choice of $h$. Select $\delta$ s.t. $\forall h$ with $||h||<\delta$ we have $||\xi(x+h)-\xi(x)||<\frac{\epsilon}{2}$ (we can do this because of continuity). Then for any such $h$ choose $0<t<1$ and small enough to have $|o(|t|)|<\frac{\epsilon}{4}||h||\cdot|t|$ and $|o(|1-t|)|<\frac{\epsilon}{4}||h||\cdot|1-t|$. Also note that $||(1-t)h||<\delta$, so this still holds $||\xi(x+(1-t)h)-\xi(x)||<\frac{\epsilon}{2}$.
Continuing the last inequality with these conditions we have $$\leq|t|\cdot||\xi(x+(1-t)h)-\xi(x)||\cdot||h||+|o(|t|)|+|o(|1-t|)|$$ $$\leq t\frac{\epsilon}{2}||h||+\frac{\epsilon}{4}||h||t+\frac{\epsilon}{4}||h||(1-t)\leq\epsilon||h||$$ So we have $$f(x+h)-f(x)-df(x,h)\in o(||h||)$$ And this proves that $df(x,h)$ is the Frechet derivative.