MATLAB: Reinforcement learning deployment in real-time system

Deep Learning Toolboxdeploymentmachine learningreinforcement learningReinforcement Learning ToolboxsimulinkSimulink Real-Time

Hello everyone,
I trained an agent on my problem, and it works really well in simulation in Simulink. I'm trying to implement the same agent in the real-time target; for that reason, I need to build my Simulink file to generate code and then upload it to my hardware. Unfortunately, I got an error in building my model. I was wondering if anyone can help me with this.
### Generating code into build folder: G:\My Drive\00Research\RL\QUBE_DC_control\Deployment\DC_motor_RL_v02_exp_grt_rtw
Dot indexing is not supported for variables of this type. Function 'evaluatePolicy.m' (#38.286.341), line 13, column 11: "coder.loadDeepLearningNetwork('agentData.mat','policy')" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment. Function 'evaluatePolicy.m' (#38.365.371), line 15, column 19: "policy" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed. Function 'evaluatePolicy.m' (#38.140.167), line 7, column 11: "localEvaluate(observation1)" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed. Function 'MATLAB Function1' (#24.45.62), line 3, column 16: "evaluatePolicy(u)" Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'DC_motor_RL_v02_exp/MATLAB Function1'
Component:MATLAB Function | Category:Coder error
### Build procedure for model: 'DC_motor_RL_v02_exp' aborted due to an error.
Errors occurred during parsing of MATLAB function 'DC_motor_RL_v02_exp/MATLAB Function1'

Best Answer

Hello,
To generate code from a trained policy, you should follow the process shown here. Note that this is a MATLAB-based workflow, i.e. you cannot generate code directly from Simulink prior to 20b, unless you import the network into your Simulink model through an S-function. Here is a video and an example that works through the workflow with GPU Coder.
In 20b, you can bring a neural network in Simulink with the deep learning block library (see video) and generate code directly from Simulink (see example).
Related Question