[Math] How to write the dual of quadratic program with positive semidefinite matrix

convex optimizationlagrange multiplierpositive-semidefinitequadratic programming

Consider the following quadratic program

$$\begin{align*}&\text{min } \frac{1}{2}b^TDb+d^Tb\\ &\text{s.t. } Ab\le b_0\end{align*}$$
where $D$ is a positive semidefinite matrix.

The Lagrangian dual function is written as follows

$$\begin{align*}L(b,\lambda)&=\frac{1}{2}b^TDb+d^Tb+\lambda^T(Ab-b_0)\\
&=\frac{1}{2}b^TDb+(d^T+\lambda^TA)b-\lambda^Tb_0 \end{align*}$$

Then I got stuck on finding the dual of the problem.

Similar background problem:

Best Answer

Informally, the approach is to swap the $\inf$, $\sup$ to convert from the primal to the dual.

With the Lagrangian above, the primal is $P: \ \inf_b \sup_{\lambda \ge 0} L(b,\lambda)$, hence we go ahead and swap the $\inf$, $\sup$ to get the dual, that is $D:\ \sup_{\lambda \ge 0} \inf_b L(b,\lambda)$.

Note that $\inf_b L(b,\lambda)$ is finite iff the convex function $b \mapsto L(b,\lambda)$ has a (global) minimiser iff there is some $b$ such that $Db + d +A^T \lambda = 0$, hence we can write $D:\ \sup_{\lambda \ge 0} \inf_{\{b | Db + d +A^T \lambda = 0\}} | L(b,\lambda)$.

The expression $L(b,\lambda)$ can be simplified using the pseudo inverse of $D$ and the fact that if $Db + d +A^T \lambda = 0$, then $b = - D^\dagger (A^T \lambda + d)$. This results in semi definite quadratic program.

Related Question