I wonder if it's possible to use a neural network in control?
Let's say that we have a deep neural network:
$$a_i = \sigma(W_i*x_i + b_i)$$
That is trained from the inputs $u(t), r(t), \Delta u(t), e(t), \Delta e(t)$ and the output $y(t)$
I have created Deeplearing2C, which create a neural network into C-code.
https://github.com/DanielMartensson/Deeplearning2C
How can we use this deep neural network for controlling an object so its output $y(t)$ follows the reference $r(t)$
Any suggestions?
Best Answer
For your general question, check the proceedings of any recent controls conference, and you'll see a plethora of ideas of how deep learning (and machine learning in general) can be used in control. There is even a new conference dedicated just to this called Learning for Dynamics and Control, you may want to watch the talks. Most of them are very good. To see two radically opposed perspectives, start with the talks of Emo Todorov and Sergey Levine.
To be a bit more specific, take for example a non-recursive net with ReLU activation functions. Each input-output path only goes through affine operations (multiplication and translation), though some paths can be shut off if they don't reach the activation threshold. In other words, you get a piece-wise affine operator from input to output. Assuming your $x_i$ are all states, this would basically be learning gain scheduling for an affine full-state feedback controller. Change your states $x_i$ to errors, integrals and derivatives of errors, and you get gain scheduling for PID. Change your activations to something nonlinear, and you get nonlinear feedback, etc..
Why would you do this?
There are many reasons. Designing a gain-scheduler by hand can be tedious. You may also rely on incorrect models for your design, and wish to refine it from data. Or you may not have good models but a lot of data (e.g. a sewage network). Or you just want to ride the wave and get some publications.
How to make it track a trajectory? I would imagine augmenting your state with your desired state, or perhaps the error directly.
Why wouldn't you do this?
Again, many reasons. It is typically really difficult if not impossible to provide any guarantees for a deep neural network. Even when they work well, you don't know under what situations they will break down. Not having failure bounds is a bit scary.
Also, it typically takes a lot of training data to get good performance on a deep net. The practical ways of generating this data will often bias the training data and aggravate over-fitting.
And while deep learning is often touted around as "magic, it just works!", in reality it typically takes a lot of trial-and-error tuning of a lot of parameters. Just as for conventional control methods. So at the end of the day, if you can do it with conventional control approaches, you (imho) probably should.
Mixing approaches
One approach I've seen a few times is to design a controller with conventional control tools, and then train a neural net whose output is added to the control block, to account for un-modeled effects. If you can bound your neural net output, you should be able to keep some of the theoretical guarantees from your conventional controller.