So I know basically what a directional derivative is and how to calculate it using the gradient vector, but I'm a bit lost on the more advanced approach of looking at it as a linear transform.
I've read that multivariable calculus is about approximating nonlinear maps with linear ones, and I know that the Jacobian is the matrix associated with the directional derivative. However, I still don't really understand the directional derivative as a linear transform.
For example, what is the input of the transform? Also, what is meant by approximating nonlinear maps by linear maps? I only know a little linear algebra so that might be why I'm so confused.
Best Answer
Given a function $f:\mathbb R^n \to \mathbb R^m$ and $h\in\mathbb R^n$. The directional derivative at $0$ in direction $h$ is defined by (if the limit exists) $$ f'(0; h) := \lim_{t\to 0} \frac{f(th) - f(0)}{t}. $$ The function, $h\mapsto f'(0; h)$ is called the Gâteaux differential or derivative. Notice that $f'(0;h)$ satisfies the linear approximation property in direction $h$: $$ \lim_{t\to 0} \frac{f(th) - f(0) - f'(0; th)}{t} = 0. $$
The function $f'(0; \cdot)$ is multiplicative by definition, that is $f'(0; th) = t f'(0; h)$. Some authors demand that $f'(0;\cdot)$ is also additive, that is $f'(0; h+k) = f'(0; h) + f'(0; k)$, thus linear. In that case we usually write $Df(0)$ or $df(0)$ for the linear map $f'(0; \cdot)$ and identify $Df(0)$ with its matrix representation, the Jacobi matrix.
If $f'(0; \cdot)$ is linear and satisfies following linear approximation property $$ \lim_{h\to 0} \frac{f(h) - f(0) - Df(0)h}{\|h\|} = 0, $$ then $Df(0)$ is called the Fréchet differential or derivative. $Df(0)$ is the unique linear map with that approximation property.