[Math] Optimal Control – difference between indirect/direct approaches

maximum-principlenumerical optimizationoptimal control

In an indirect method, my understanding is we convert the Continuous Optimal Control problem to a 2-point boundary problem by using initial conditions on the states and terminal conditions on the costates.

Then there are the direct methods based on input and/or control discretization. In direct collocation, both states and controls are discretized and a single NonLinear Program is solved.

3 questions then:

  1. Is the control trajectory found through indirect/direct approaches is a locally optimal one? (I think it is using the maximum principle so I don't see why this can be a global optimal)
  2. How exactly is direct collocation is different then discretizing states/controls/time?
  3. Lasty, why is indirect apprach is also called "first optimize, then discretize", where is the discretization part?

Best Answer

  1. solution (if one found) of the NLP in direct methods are local optimal but not neccesary global optimal (except for some special types of NLPs, e.g. convex NLP).

for the indirect approach, you solve as you already mentioned, the neccessary condition for optimality of the optimal control problem (minimum principle ,...). This forms a boundary value problem (BVP) which is often nonlinear and has several local optimal solutions (optimal state and optimal costate). The problem here is, you can solve the BVP but you obtain only ONE local optimal solution (you don't know if its global optimal). In order to find the global optimal one, you would have to determine all local optimal solutions and then use the HJB equation to check, which one of the local opt. solutions are also global optimal.

In generall, the solution depends on your initial guess (for solving numerically). For two different initial guesses you might obtain two different solutions. This holds for both methods (direct, indirect)

  1. i don't understand your question! Please reformulate!

  2. i am not sure why they call it like that. The main difference of direct and indirect methods are:

general optimal control: minimize a functional (performance measure) subject to constraints/bounds and of course the dynamics

direct method: solve DIRECTLY this optimal control problem as it is written there.

indirect method: solve the neccessary condition (BVP) which INDIRECTLY represents the original optimal control problem. you just rewrite the optimal control problem (which is an optimization problem) in another optimization problem. Both problems

then, you can use the same methods (e.g. collocation, shooting) for both direct and indirect methods. you only solve different sytems of ODE with different constraints.

Related Question