[Math] In Lagrange Multiplier, why level curves of $f$ and $g$ are tangent to each other

lagrange multipliermultivariable-calculus

In Lagrange multiplier method, e.g. optimize a function $f(x_1, \dots, x_n)$ under a constraint $g(x_1, \dots, x_n) = 0$. There is a fact that $\nabla f$ is parallel to $\nabla g$ which is given rise from the level curves of $f$ and $g$ are tangent to each other (i.e. there tangent lines are parallel, then because gradient and tangent of level curve are orthogonal implies the fact above) at the points when $f$ optimized under constraint $g$.

The only part I don't have intuitive understanding is that why level curves of $f$ and $g$ are tangent to each other at where $f$ optimized under $g$.

Best Answer

The level curves of $f$ represent single values of $f$ that increase in a direction parallel to the gradient. This means that, given a level curve that does not represent a local maximum, there is another level curve nearby whose value for $f$ is greater than the first curve.

Imagine $g$ as a curve that cuts through a level curve of $f$ at a point $p$. Since $g$ cuts the curve, there are level curves of $f$ on either side of $p$ that also intersect with $g$. Therefore, we can choose another level curve with a greater value for $f$ than the one that contains $p$ and so the maximum cannot occur on that curve.

Therefore, to maximize $f$, we choose level curves in the direction of increase until we can go no further which will occur when the level curve of $f$ is tangent to $g$.