In an optimization problem, I have an inequality constraint, say
$\begin{array}{c}
{\min\limits_x~} c(x)\\
{s.t.~}g(x)\le 0
\end{array}$
The function $g(x)$ in general is unknown. So, numerical perturbation is necessary if the solver is gradient-based.
Now, I can pass it to a universal nonlinear optimization solver, e.g., fmincon in MATLAB. However, what if I convert the constraint to
$\begin{array}{c}
{\min\limits_x~} c(x)\\
{s.t.~}\max(g(x),0) = 0
\end{array}$
and pass this equality constraint to the same optimization solver?
I would expect to have the same results in both cases (Please ignore the numerical tolerance for simplicity). But does this conversion bring any disadvantage or advantage theoretically? Could there be any major impact?
Best Answer
If the solver is smart, it'll actually convert $\max(g(x),0)=0$ back into $g(x)\leq 0$. Basically, the $\max$ function is nondifferentiable, which makes it much more difficult to work with numerically since we typically need derivatives for fast optimization algorithms. As such, a solver with a good preprocessor will take $\max(g(x),0)=0$ and turn it into \begin{align*} y =& 0\\ y\geq& g(x)\\ y\geq& 0 \end{align*} which then gets further processed into $$ 0\geq g(x) $$ which you started with. Or, at least, that's one way to process it. There's probably more.
Anyway, in answer to your original question. No. Your transformation makes things much more difficult numerically. Again, the $\max$ function is nondifferentiable, so it's hard to use fast algorithms to work with it. In addition, if $g$ were convex, you just reformulated a nice convex constraint into something that's hard to work with. Now, unless $g$ is affine, it's likely that the solver is going to add a slack variable anyway and reformulate your problem as $$ \min\limits_{x\in X} \{c(x) : y = g(x), y\leq 0\} $$ It'll do this since most algorithms to handle inequalities need to figure out how far we can traverse before becoming infeasible. For a nonaffine $g$, this is hard to determine, so we just add a slack variable and then handle the equality constraint. As such, the real question then becomes whether or not we can effectively solve the linear systems that involve $g^\prime(x)$, which arise when attacking the KKT conditions using something like a Newton method. Basically, algorithms like SQP. In any case, if you want to speed up the computation, likely the biggest speedup is focusing on quickly solving these linear systems.