In calculus II we were introduced to a bunch of new derivatives: the gradient, the derivative $D=\begin{bmatrix} \partial_{x_1} \\ \partial_{x_2} \\ \vdots \\ \partial_{x_n}\end{bmatrix}$, the Jacobian, the Hessian, the total differential, the directional derivative, the partial derivative, and something called a Frechet derivative (that one was only mentioned in passing).
I can apply the formulas to calculate these things, but what exactly are they? And how do they relate to each other?
For instance, the derivative of a function $f: \Bbb R \to \Bbb R$ gives the slope of the line tangent to $f$. Which one of the above gives you, for instance, the "slope" (I don't even know what to call a $2$-D slope) of a function $g: \Bbb R^2 \to \Bbb R$? I know that the partial derivatives give you the slope in the $x$, $y$, etc directions, but then what do the others do?
And how do they relate to each other? For instance, how does $D$ relate to say the directional derivative $\partial_{\vec v}$?
Thanks.
Best Answer
Let $f:A\subseteq\Bbb R^n\to \Bbb R$ and $\mathbf g:B\subseteq\Bbb R^m\to \Bbb R^n$ where $A,B$ are open subsets.
The directional derivative of $f$ is defined exactly the same, but will be scalar-valued. Alternatively, the directional derivative of $f$ can be defined as follows. Let $\gamma:\Bbb R\to\Bbb R^n$ be given by $\gamma(t)=\mathbf p + t\mathbf v$. Then we can define the directional derivative by $$D_{\mathbf v} f(\mathbf p) := (f\circ \gamma)'(0)$$ This definition has the benefit of being defined in terms of a purely scalar derivative. Confirm for yourself that these are equivalent definitions.
The partial derivative of $f$ is defined exactly the same, but will be scalar-valued.
It should be noted that, like all of these tougher-looking definitions, this one simplifies in the case of Cartesian coordinates. This time to $$\nabla \cdot \mathbf g(\mathbf p) = \sum_i^n\partial_i \mathbf g_i(\mathbf p)$$ where $\mathbf g_i$ is the $i$th component of the vector function $\mathbf g$.
And again, this definition simplifies for Cartesian coordinates to $$\nabla \times \mathbf g(\mathbf p) = \big(\partial_2\mathbf g_3(\mathbf p) - \partial_3\mathbf g_2(\mathbf p), \partial_3\mathbf g_1(\mathbf p) - \partial_1\mathbf g_3(\mathbf p),\partial_1\mathbf g_2(\mathbf p) - \partial_2\mathbf g_1(\mathbf p)\big)$$
The derivative of $f$ is obtained by setting $n=1$, where $\|\cdot\|_{\Bbb R^1}$ is simply the absolute value function. Note that if $Df(\mathbf p)$ exists then its matrix representation is $[\nabla f(\mathbf p)]^T$.
As for the connection between these, beyond what I've already stated above, the total derivative really is the one that contains all of the information of the others. The total derivative of $f$ has matrix representation $[\nabla f]^T$, whose coordinates in the Cartesian coordinate system are exactly the partial derivatives, and it has the property that $$Df(\mathbf p)(\mathbf v) = D_{\mathbf v}f(\mathbf p)$$
Very similar statements hold for $D\mathbf g$. But also $D\mathbf g$ encodes all of the information contained in the divergence and curl. $\nabla\cdot \mathbf g(\mathbf p)$ is just the trace of $D\mathbf g(\mathbf p)$ and $\nabla \times \mathbf g(\mathbf p)$ has the exact same components as $J\mathbf g(\mathbf p)-(J\mathbf g(\mathbf p))^T$ where $J\mathbf g(\mathbf p)$ is the Jacobian of $\mathbf g$ at $\mathbf p$ -- which as stated above is just the matrix representation of $D\mathbf g(\mathbf p)$.
The Fréchet derivative is defined on Banach spaces, which as you may or may not be aware are generally infinite dimensional space and thus require a little bit more care. In fact Banach spaces don't even have a notion of an inner product to exploit. But they do have a norm. Thus maybe it won't surprise you that the definition is given as follows. Let $V,W$ be Banach spaces and $f:V\to W$. Then $f$ is Fréchet differentiable at the point $p$ if there exists a bounded linear function $L:V\to W$ such that $$\lim_{h\to 0}\frac{\|f(p+h)-f(p)-L(h)\|_W}{\|h\|_V}=0$$ If $L$ exists then it's called the Fréchet derivative and is denoted $Df(p)$.
Since you asked about the tangent plane to a surface, I'll share this as well:
Let $\mathbf x: A \subseteq \Bbb R^2 \to S \subseteq \Bbb R^n$ parametrize a surface. Let $\mathbf q\in A$ and set $\mathbf p=\mathbf x(\mathbf q)$. Then the vector $D_{\mathbf w}\mathbf x(\mathbf q)$ is a tangent vector to $S$ at $\mathbf p$. The set of all tangent vectors (in this case, the tangent plane) at $\mathbf p$ is the tangent space to $S$ at $\mathbf p$. It's denoted $T_{\mathbf p}$.