Hi I have this question about Lagrange multipliers and specifically when there are 2 constraints given. The standard answer to this question uses the lagrangian and 2 constraints with 2 extra variables (lambda 1 and lambda 2). I understand this method completely fine, but when I first attempted this problem, I tried combining the 2 constraints given into one constraint (i.e. putting x=2z+3 into the other constraint) and then using Lagrange multipliers with only 1 constraint. However, this gave a completely different result! (The standard solution was sqrt(2) but I got 1 with my method). Can anyone explain why these 2 methods give different results?
Question about Lagrange multipliers and combining constraints
constraintslagrange multipliermultivariable-calculusoptimizationsystems of equations
Related Solutions
What goes wrong is that the minimum might happen to be at a point where a derivative might not exist. Try e.g. minimize $y$ subject to $g(x,y) = y - |x| = 0$. The minimum is at $(0,0)$, where $\frac{\partial g}{\partial x}$ does not exist.
I think the difficulty with the subdifferential approach you describe lies in finding critical points. For instance, with your example, solving the equations you get by setting partials to $0$ yields only $x_1 = | x_2 |$ and $\lambda = -1$. This set of equations is too undetermined to yield a solution. As you point out, if you were given a solution $x_1 = x_2 = 0$, $\lambda = -1$, you could verify that it is a critical point with the subdifferential approach, but that's quite different from deriving $x_1 = x_2 = 0$, $\lambda = -1$ as a critical point.
Remember, though, that the definition of a critical point of a function includes points at which the derivative fails to exist. So when you're constructing your set of critical points using Lagrange multipliers, look not only for points at which the partials are zero but also for points at which a partial does not exist. For instance, it's clear in your example that the only point at which $\frac{\partial L}{\partial x_2}$ fails to exist is $x_2 = 0$. (You also get $\lambda = 0$ from $\frac{\partial L}{\partial x_2}= 0$, but this is inconsistent with the $\lambda = -1$ from $\frac{\partial L}{\partial x_1}= 0$.) Including that with your equations from $\frac{\partial L}{\partial x_1} = 0$ and $\frac{\partial L}{\partial \lambda} = 0$ quickly gets you the solution $x_1 = x_2 = 0$, $\lambda = -1$.
In fact, finding points where a partial does not exist is generally not any harder (and is sometimes easier) than finding points where a partial is zero. There aren't many ways for a point in the domain of a continuous function to have a partial that fails to exist. (Besides the absolute value situation, you could have division by zero with the partial from an expression like $x^p$, with $0 < p < 1$, in the function. Perhaps there are some others I'm forgetting - maybe others reading this can supply them. There are also some pathological cases, like the continuous but nowhere differentiable Weierstrass function, but those don't generally show up in practice.)
And, of course, if you have more than just a few variables, you don't want to use Lagrange multipliers anyway: Solving a large number of nonlinear equations is just too difficult. In that case, you will probably want an iterative nonlinear optimization method.
Best Answer
Notice that replacing two constraints with one is not valid. Just to make this as straightforward as possible, consider the system of linear equations $x+y=0$ and $x-y=0$. The only solution, of course, is $x=y=0$. However, if we add the equations, eliminating $y$, we obtain $2x=0$, which is an entire line. Obviously, this alone is not enough information.
When you combined your two constraints, you went from a curve of eligible points to a surface of eligible points (a cylinder over an ellipse in the $yz$-plane). This allows too many eligible points.