Lagrange multiplier parallel slope intuition confusion

lagrange multipliermultivariable-calculus

One specific explanation of intuition regarding Lagrange multipliers that I cannot wrap my head around is presented in this Medium article and in Jeff Peterson's answer here, both very popular and I think I'm missing something essential in my understanding.

The Medium article explains as this:

In reference to the following image: enter image description here

Imagine hiking from left to right on the constraint line. As we gain elevation, we walk through various level curves of f. I’ve marked two
in the picture. At each level curve, imagine checking its slope — that
is, the slope of a tangent line to it — and comparing that to the
slope on the constraint where we’re standing.

If our slope is greater than the level curve, we can reach a higher
point on the hill if we keep moving right. If our slope is less than
the level curve — say, toward the right where our constraint line is
declining — we need to move backward to the left to reach a higher
point.

When we reach a point where the slope of the constraint line just
equals the slope of the level curve, we’ve moved as high as we can.
That is, we’ve reached our constrained maximum. Any movement from that
point will take us downhill. In the figure, this point is marked with
a large arrow pointing toward the peak.

At that point, the level curve f = a2 and the constraint have the same
slope. That means they’re parallel and point in the same direction.
But as we saw above, gradients are always perpendicular to level
curves. So if these two curves are parallel, their gradients must also
be parallel.

This argument doesn't quite make sense to me. First, I don't understand what it would mean to check the slope of the level curve. We have a tangent line in $\mathbb{R}^3$ that is tangent to a given level curve at some point. But then how does this relate to one slope being "greater" than another when we're comparing vectors in $\mathbb{R}^3$. I suppose you could look at the intersection and compute the tangent line of each level curve at the point of intersection, but it still doesn't allow us to compare to say one is greater/less.

Similarly, in Jeff Peterson's explanation, he explains:
enter image description here

In this figure, 𝑔(𝑥,𝑦)=𝑐 is represented as linear in x and y and
defines the coordinates x and y on 𝑓(𝑥,𝑦) where we must look for
the maximum. The intersection of these surfaces forms the cyan (light
blue) line and its maximum is exactly where the contours of 𝑓(𝑥,𝑦)
are parallel to the plane.

It seems like this is a common way of thinking about Lagrange multipliers intuitively but similarly in this post I'm so confused on the notion of how there are contours parallel to the plane when it seems like the blue plane would lie perpendicular to the contour lines.

Am I thinking about all this wrong? I have somewhat of an intuition of Lagrange multipliers from Wikipedia but I'd like to understand what I'm missing here in these explanations.

Best Answer

Some ideas.

Assuming $f, g$ are nice functions and that $g(x_1,x_2)$ has inverse inside a convenient ball $|x-\bar x| < \rho$ we have inside this ball $g(x_1,x_2) = 0 \Leftrightarrow x_1 = \phi(x_2)$

So the problems

$$ \min(\max)_{x_1,x_2} f(x_1,x_2)\ \ \mbox{s. t.}\ \ g(x_1,x_2) = 0 $$

and

$$ \min(\max)_{x_1,x_2} f(x_1,x_2)\ \ \mbox{s. t.}\ \ x_1-\phi(x_2) = 0 $$

and

$$ \min(\max)_{x_2} f(\phi(x_2),x_2) $$

are equivalent.

For the last we have the stationary points as

$$ \phi'f_{x_1}+f_{x_2} = 0 $$

and the stationary points for $f(x_1,x_2) + \lambda(x_1-\phi(x_2))$ are obtained as a solution for

$$ \nabla f+\lambda\nabla(x_1-\phi(x_2)) = \left\{\begin{array}{rcl}f_{x_1}+\lambda = 0\\ f_{x_2}-\lambda \phi'(x_2) = 0\end{array}\right. $$

After the $\lambda$ elimination we have again

$$ \phi'f_{x_1}+f_{x_2} = 0 $$

so the set of solutions for

$$ \min(\max) f(x_1,x_2)\ \ \mbox{s. t.}\ \ g(x_1,x_2) = 0 $$

and for

$$ \nabla f(x) + \lambda \nabla g(x) = 0 $$

have common points. Those points require further qualification.

Related Question