I am implementing the Support Vector Regression (SVR) algorithm by means of quadratic programming. In order to do that, I am using an optimization library that contains a quadratic solver based on the Goldfarb Idnani active set quadratic optimization method.
However, the library seems not to be working the way I was expecting. Once the optimization process is finished, not all the constraints are active, so there are some variables for which the value is out of the valid range of values in an SVR problem.
I implemented SVR in the past in Matlab by means of the quadprog method, and I did not have such issues. Of course, I reviewed and debugged my own code.
My question is: is it normal that, after finishing the Goldfarb Idnani algorithm, some of the constraints are not active? Or should I assume that the library is not working properly?
PS: I have read the original paper[1], and from it I guess that all constraints should be active at the end of the process.
[1] "A numerically stable dual method for solving strictly convex quadratic programs", D. Goldfarb, A. Idnani, Mathematical Programming 27, pp. 1–33, 1983. Available at http://www.javaquant.net/papers/goldfarbidnani.pdf.
Best Answer
If hard constraints of the optimization problem are violated in the solution there is most definitely a problem in your implementation. A solution is required to adhere to all constraints. Note that this does not mean the solution must be somewhere on the boundary of the feasible region (in contrast to linear programming).