I've seen a few questions about the subject but still couldn't figure it out.
I don't fully understand why the Lagrange multipliers always relate to tangent contour lines between the specified function and the constraint. What if the maximum or the minimum of the function intersects the constraint but is not tangent to it?
Below I've drawn an example which summarizes my struggle with the concept. The circle is the constraint, and the lines are the contour with values.
Function:
Best Answer
Assuming the function indicated here waves nicely and sine-like in the way hinted at in the drawing (say we have $f(x,y)=\cos(x+y)+3$), the Lagrange multiplier method will find all the red dots in the given image.
Let's also say, just to be concrete about our niceness assumptions, that the constraint equation is $x^2+y^2-\left(\frac{3\pi}{2\sqrt2}\right)^2=0$, which makes $g(x,y)=x^2+y^2-\left(\frac{3\pi}{2\sqrt2}\right)^2$ the constraint function. This will indeed make the green contour lines at level $f(x,y)=2$ perfectly tangent to the constraint circle.
At points marked $B$ the gradient of $f$ is $0$, so clearly the gradient is orthogonal to the circle, and therefore parallel to the normal vector to the circle (the normal vector to the circle is the same as the gradient of $g$, which is what the Lagrange multiplier method usually uses).
We don't really need the multiplier method to find these points, as the gradient of $f$is actually $0$, and we can use more direct methods. But the multiplier method will find them for us.
The two points marked $A$ (which is what the Lagrange multiplier method specialices at) will be found because the gradient of $f$ is pointing orthogonally to the green 2-lines. And therefore, since the circle is tangent to the 2-lines at those points, the gradient of the function is also orthogonal to the circle (and parallel to the gradient of the constraint function).
These are the points where we need something like the multiplier method to help us, because the extremal nature of these points, as opposed to the $B$ points, is the result of a more intricate interplay between the constraint and the function. It isn't just an extreme point of $f$ that happens to lie on the circle, it is an extreme point along the constraint because the constraint "turns around" compared to the gradient of $f$.
If we move into a realm of real-world analogy, consider a mountain hiker (who never actually stops, so her path through the terrain has continuous, non-zero derivative everywhere). If she logs her altitude continuously, and looks at the log afterwards, what kind of tops and bottoms will she see on the graph?
Clearly, whenever she reached the highest peak of a mountain or the lowest point of a valley, those will show up. These are the points where the gradient of the true altitude of the regions is zero.
But she will also see a top whenever she was walking upwards, and then started walking downwards (or vice versa), even if that didn't happen anywhere near a peak (or valley). And at the exact turnaround moment, her walking trajectory will have been parallel to the level lines of the terrain.
If she draws her walk on a map, the Lagrange multiplier method will help her locate all of these points, but she doesn't really need the method to see the peaks and valleys; those are already readily visible. But it is the turnaround points, where she walked momentarily parallel to the level lines, that makes the multiplier method special.