What is the point of KKT conditions for constrained optimization?
In other words, how is the best way to use them.
I have seen examples in different contexts, but miss a short overview of the procedure, in like one or two sentences.
Should we use them to find the optimal solution of a constrained problem?
The reason I am very confused is that one of conditions in KKT, already requires the constraints of the original problem to hold. The question is if we knew how to impose constraints in first place, then why look at KKT conditions?
Or should we use another one of KKT conditions first, i.e. only set the gradient of Lagrangian to zero and extract the solutions from that, and then check if the inequality and equality constraints hold?
I deeply appreciate if you could clarify.
Best Answer
Since it doesn't seem that anybody is giving an answer I will slightly elaborate on my comments above. The first thing to point out is that KKT conditions don't give a "procedure" as you're question implies. Rather, KKT conditions give a "target" for procedures to move towards.
KKT conditions are primarily a set of necessary conditions for optimality of (constrained) optimization problems. This means that if a solution does NOT satisfy the conditions, we know it is NOT optimal. In particular cases, the KKT conditions are stronger and are necessary and sufficient (e.g., Type 1 invex functions). In these cases, if a solution satisfies the system of KKT conditions it is globally optimal.
So what do the KKT equations do for us? By giving us a system of equations, we can attempt to find a solution to them. Typically, we can't solve these equations analytically, so we use numerical methods to solve them (e.g., sequential quadratic programming).
If you have specific questions about numerical (or exact) methods in given contexts, I'd suggest asking a new question with those details.