I know this is a topic covered on the internet frequently, but I still have further questions regarding visualization. I last took calculus some time ago, but am struggling with visualizations.
Assuming $f(x,y)$ is some surface in 3D space. Now, let's say we take the gradient at point $(x_1, y_1)$. $\nabla f$ is then some two-dimensional vector $\langle \frac {\partial f}{\partial x},\frac{\partial f}{\partial y}\rangle$ and evaluated at $x_1$ and $y_1$. This two-dimensional vector symbolizes both how much change we have with respect to the x-axis and how much change we have with respect to the y-axis. I now have a couple points that I'm struggling (for whatever reason, to figure out)
1) Is it safe to say that the magnitude of the gradient vector at $(x_1, y_1)$ is the slope of the tangent plane to the surface at $f(x_1, y_1)$?
2) How is it that I read the gradient both is normal to the surface, but also points in the direction of maximum increase of $f$? I guess this is the problem I'm having, visualizing what the gradient vector looks like.
Best Answer
Your function $f:\mathbb{R}^2\to\mathbb{R}$ gives a surface in $\mathbb{R}^3$. This is the subset of $\mathbb{R}^3$ given by $\{(x,y,f(x,y)) \ ; x,y\in\mathbb{R}\}$ which is equal to $$\left\{(x,y,z)\in\mathbb{R}^3 \ ;\ f(x,y)-z=0\right\}$$ The gradient of the function $g(x,y,z)=f(x,y)-z=0$ is the normal vector you are referring to. The gradient is $\left(\frac{\partial f}{\partial x},\frac{\partial f}{\partial y},-1\right).$ At each point $(x_0,y_0,z_0)$ on your surface, this vector evaluated at $(x_0,y_0,z_0)$ gives a normal vector for the tangent plane to the surface at $(x_0,y_0,z_0)$.
To address your second question, recall that the directional derivative of $f(x,y)$ in the direction of a line in the $xy$-plane gives the slope of the line obtained by extending the line in the $xy$-plane in the $z$-direction and intersecting the surface, as shown below
(source: buffalo.edu)
The gradient of $f$ is $(\frac{\partial f}{\partial x},\frac{\partial f}{\partial y})$, which, at each point, gives a line passing through the origin in $\mathbb{R}^2$. It is this line (i.e. direction) for which the directional derivative returns the largest value. That is, the slope of the blue line is the greatest when your direction in the $xy$-plane is $(\frac{\partial f}{\partial x},\frac{\partial f}{\partial y})$.
In the animation notice how as the black line changes, the slope of the blue line (which is the directional derivative in the direction of the black line) changes. The green line in the xy-plane is the line formed by the gradient of f(x,y) evaluated at the yellow point. Notice that when the directional derivative is taken in the direction of this line (the green line) the blue line is the steepest.