I'm working on a project that needs to solve a constraint optimization function. Currently, I'm using Knitro solver and it needs to calculate the the hessian of the lagrangian at x and lambda. I don't understand how to calculate the hessian. The constraint optimization problem is as follows:
$\text{minimize}_x 100 -(x_2 – x^2_1)^2 + (1-x_1)^2$
subject to $1\le x_1x_2$, $
0\le x_1,x_2$,$x_1\le0.5$.
The function to calculate the hessian is (it seems that lambda is given):
$ t = x_2 – x_1x_1;$
$ h_1 = (-400.0 t) + (800.0 x_1x_1) + 2.0$,
$ h_2 = (-400.0 x_1) + \lambda_1$,
$ h_3 = 200.0 + \lambda_2 2.0)$
Could you please tell me how to get $h_1,h_2,h_3$ ? Thank you very much.
[Math] How to calculate the Hessian of the Lagrangian at x and lambda
multivariable-calculusnonlinear optimization
Related Solutions
Seems that you're almost there: $$ (x_1,x_2,x_3) = \left(2\sqrt{\frac{2}{5}},0,\sqrt{\frac{2}{5}}\right) \quad \Longrightarrow \quad f(x_1,x_2,x_3)=\sqrt{10}\\ (x_1,x_2,x_3) = \left(-2\sqrt{\frac{2}{5}},0,-\sqrt{\frac{2}{5}}\right) \quad \Longrightarrow \quad f(x_1,x_2,x_3)=-\sqrt{10}$$ The former outcome must be the maximum and the latter outcome must be the minimum. No need at all to calculate the multiplier $\lambda_1$, I think (but my knowledge about Lagrange multipliers is rusty).
We want to minimize $x_1+x_2$ and $x_1,x_2 \ge 0$, so we have to get close to origin as much as possible. Consider the problems in $x_1-x_2$ plane, first we have to find the feasible region (I am assuming $a_1,a_2\ge0$):
The blue curve is the boundary of the first inequality $x_1 x_2 \ge a_1$, the feasible region is the region above this curve. The dashed-line orange curves are the second inequality $x_1x_2\ge ya_2$ for different values of $y$, again the area above these curves are the feasible region. This means if $y\le\frac{a_1}{a_2}$, we can ignore the second inequality, otherwise (if $y\ge\frac{a_1}{a_2}$) we can ignore the first inequality. The purple vector shows the direction of movement of curves as we increase $y$. Then we have the third inequality $x_1 \ge y$ which has the boundary of green line, our answer is in the right hand side of green line. And finally we have the forth inequality $x_2 \le y$ with boundary of grey line, our answer is in lower half of this line (below grey line).
With these information at hand, we see that the inequality $x_1\ge y$ must become active at solution point, this means the slackness condition $\lambda_4(y-x_1)=0$ is equivalent to $x_1=y$. Substituting this in the primal, we get three inequalities $x_2 \ge \frac{a_1}{y}$, $y\ge x_2$ and $x_2 \ge a_2$ and the objective is $y+x_2=x_1+x_2$.
Back to primal. Now consider on one hand we have we have $y\ge x_2$ and $x_2 \ge \frac{a_1}{y}$ which means $y \ge \frac{a_1}{y}$ or $y \ge \sqrt{a_1}$. On the other hand we have $y\ge x_2$ and $x_2 \ge a_2$which means $y\ge a_2$. Thus we obtain two main conditions that solves everything: $y \ge \sqrt{a_1}$ and $y\ge a_2$.
Finally if $a_2 \le \sqrt{a_1}$ the solution is $x_1=y=x_2=\sqrt{a_1}$. Otherwise if $\sqrt{a_1} \le a_2$ the solution is $x_1=x_2=y=a_2$. And now we can see the funny part, both $x_1 \ge y$ and $y \ge x_2$ are tight so from start we could have considered the slackness conditions $\lambda_3(y-x_1)=0$ and $\lambda_4(x_2-y)=0$ to be active i.e. $x_1=x_2=y$ to be true and get the solution.
Best Answer
You don't need Lagrange multipliers to deal with this problem.
I read your constraints as $$0\leq x_1\leq {1\over2},\ x_2\geq0,\ x_1 x_2\geq1.$$ It follows that the feasible domain $B$ is bounded by the halfline $h_1: \ x_1={1\over2}$ starting upwards at the point $P:=({1\over2},2)$ and by a steep arc $h_2$ of the hyperbola $x_1 x_2=1$ starting at $P$ as well. The gradient of $f$ computes to $$\nabla f(x_1,x_2)=(\ldots, -2x_2+2x_1^2)\ .$$ It follows that $f$ has no stationary point in the interior of $B$, as $x_1\leq{1\over2}$ and $x_2\geq2$ there.
Along $h_1$ we have to consider the pullback $$\phi_1(x_2):=f\bigl({1\over2},x_2\bigr)=-x_2^2 +{1\over2} x_2+101.0625 $$ which obviously is monotonically decreasing to $-\infty$ going up along $h_1$. Analogously we can study the pullback along $h_2$, which is more complicated: $$\phi(x_2)=f\bigl({1\over x_2},x_2\bigr)=101-x_2^2 -{1\over x_2^4}+{1\over x_2^2}\ .$$ It should be possible to show that $\phi_2$ decreases monotonically to $-\infty$ as well when going upwards along $h_2$.
Since in fact we have $\lim_{x_2\to\infty}f(x_1,x_2)=-\infty$ uniformly in $x_1$ for $0\leq x_1\leq{1\over2}$ it follows that your function assumes its maximum at the point $P$ and is unbounded from below on the feasible domain $B$.