I am having trouble understanding visually what a gradient is. My understanding is it is a generalisation of tangential slopes to higher dimensions and gives the direction of steepest ascent. There are 4 different pictures I have:
1) From Khan Academy. How does it make sense to have a 2d gradient when your function is 3d? And shouldn't it be tangential to the function? The only way this makes sense to me is if you consider it as the projection of the tangential gradient vector onto the x-y plane. Is this right?
2) From a Medium article explaining Lagrange Multipliers. I understand the gradient for f but I'm not able to understand the gradient of g visually. It is a plane right?
3) I have plotted some samples from MacOS Grapher. I'm assuming Vector Field Cartesian form is the Gradient Vector. Why are the arrows going inward? Shouldn't they be tangential to the curve?
4) Finally, I have drawn some 3d curve. Could you tell me which gradient is correct? A, B or C?
Any help appreciated. Sorry for the lengthy post but I have been breaking my head on this for a while. Thanks in advance.
Edit: Changed Legrande to Lagrange.
Best Answer
The function is two-dimensional, as in, it takes in two real numbers as input. The domain of the function is the plane, so the gradient also lives in the plane. Yes, the graph is drawn in 3-dimensional space, but that's because you need three dimensions to accurately represent two-dimensional input and one-dimensional output.
If you instead visualise the function as simply a number for each point in the plane (it's boring, and not easy to draw, but in some sense truer to what's actually going on), then maybe this intuition works a little better. Yes, in your graph example, the gradient is sort of a projection of the steepest tangent vector at each point, but really, it's the other way around.
It's Lagrange multipliers, not Legrande. And it's not so strange that you don't really understand the drawing, because I barely do. At least without an accompanying explanation. They seem to have drawn the graph of a function $f$ (as in 1.), and the constraint $g(x_1, x_2) = c$, but again, the arrows really should be in the $x_1x_2$-plane, not tangent to the graphs. And in the plane, $g(x_1, x_2) = c$ is a line, the gradient of $g$ is orthogonal to that line (see 4.), and when the gradient of $f$ is parallel to the gradient of $g$, then that means that moving along $g$ doesn't change the value of $f$, which again is a characteristic of extremal points. This is the main idea behind Lagrange multipliers.
Again, you're mixing 2d and 3d. What you have is the graph of the function $z(x, y) = -x^2-y^2$. It's a function with a two-dimensional domain, and thus the gradient ought to live in the $xy$-plane. However, your vectors are in 3d. So what's going on there?
Note that the $x$ and $y$ components of your vectors do corespond to the gradient of the function $z$, so if you take away the $z$-component from your vectors, they do indeed, at each point, point in the direction where the graph increases the most: straight in towards $(0,0)$.
The same graph can also be seen as the level set $g(x, y, z) = 0$ for $g(x, y, z) = -x^2-y^2-z$. Taking the gradient of this function $g$, we get $[-2x, -2y, -1]$. Note that if you plot this vector field (change the sign of the $z$ component compared to your drawing), then the vectors are orthogonal to the surface (see 4.), and pointing inwards because that's the direction the surface moves if you increase the $0$ on the right-hand side of $g(x, y, z) = 0$.
Is this the level surface of some function in three variables, or is it the graph of a function in two variables? If it is the graph of a function in two variables, I believe you already know my answer: A is the closest, but needs to be projected down into the $xy$-plane.
On the other hand, if we have some function $f$ of three variables, and this graph is the level set $f(x, y, z) = c$ for some constant $c$, then B and C are actually gradients of $f$. They are, in a sense, both correct: if $f(x, y, z) = c$ gives the surface and $B$ is the gradient of $f$, then $-f(x, y, z) = -c$ gives the same surface, and $C$ is the gradient of $-f$. At least if you move it so that it starts at the graph.
This is because the gradient points toward the direction of greatest increase of a function. And moving along a level set doesn't change the value of a function at all. So assuming differentiability, moving along the level set and moving in the direction of greatest increase must be orthogonal. So the gradient of $f$ is orthogonal to any of its level sets. The direction of the gradient tells you in which direction the level set will move if you increase the constant $c$, and the size of the gradient tells you how fast the surface will move at that point.
Of course, a function like this $f$ will likely have gradients everywhere in space, not just on the level surface (just like functions that take two inputs have a gradient everywhere in the plane). These other gradients are orthogonal to other level sets.