[Math] Constrained optimization: equality constraint

nonlinear optimizationnumerical methodsnumerical optimizationoptimization

I have this very general problem (for $n>2$):

$$
\begin{align}
& \max Z = f(x_1,\ldots ,x_n) \\[10pt]
\text{s.t. } & \sum_{i=1}^{n} x_i = B \\[10pt]
& x_i \geq 0
\end{align}
$$

Assume that the function $f(x_1,\ldots,x_n)$ is not a "nice" function, non-linear, non-convex, hardly differentiable. Therefore, in general I cannot use the Lagrangian multipliers method.
For finding the $x_i$'s, I am thinking to numerical optimization approaches.

My questions are:

1) Is there a common, smart and effective way to deal with such an equality constraint in numerical optimization?

2) Am I missing something? Do you have any other direction to suggest?

Thanks

Best Answer

The methods to be used will be highly dependent on the character of $f$. If it is non-convex, there are many such algorithms; see this MO post for instance.

You might also wish to look into evolutionary algorithms, such as genetic algorithms and simulated annealing. These algorithms are often much slower, but have the feature that they can sometimes "bump" you out of local extrema. They are also fairly easy to implement.

You can also hybridize approaches: combine an evolutionary algorithm with a standard convex optimization approach on a locally convex subdomain.

Finally, with an equality constraint, you essentially reduce the dimensionality of your problem by 1. That is, one variable is completely determined by the others.

$$x_k = B - \sum_{i=1,\ i\neq k}^n x_i.$$

And then, depending on the character of your function, you might be able to use any sort of algorithm.

But to answer your questions, 1.) there are many common and smart ways, but they depend on your function $f$. I would start with the most simple, conventional approach, and then see whether it is effective. 2.) I don't think you're missing anything. Numerical optimization is a big topic and there are many ways to go about it.

Related Question