MATLAB: How to specify multiple, heterogeneous actions for the rl.env.MATLABEnvironment of the Reinforcement learning toolbox or another way, if there is one

Question

How would I go about defining very different actions for ActionInfo? It's a "specification object" but I'm not sure where the definition is for this. The documentation and examples are all for a single array with actions. This may mean the agent would have a much harder time optimizing these actions due to redundant action spaces due to the same limit for every action. See example below for 2 very different actions that I might need:

% Define action info

ActionInfo(1) = rlNumericSpec([1 1, LowerLimit 4, UpperLimit 10]);

ActionInfo(1).Name = 'speed';

ActionInfo(2) = rlNumericSpec([1 1, LowerLimit 3000, UpperLimit 10000]);

ActionInfo(2).Name = 'distance';

Thanks for your help and this awesome toolbox!

Answer 1

Hello,

I probably don't understand your objective but the two actions you mention above (distance and speed) are still scalars. What difference would it make to split them like that? Do you want different layers for each of the two for feature extraction?

If you want two scalar actions with different limits, you can do

ActionInfo = rlNumericSpec([2 1],'LowerLimit',[4;3000],'UpperLimit',[10;10000])

If you want to see how to use rlNumericSpec with different types of inputs (e.g. images and scalars) this example that uses heterogeneous observations (scalars and images) may be helpful. Check obsInfo to see how it's set up or type

edit rl.env.AbstractSimplePendlumWithImage

to look into how the predefined environment is created (see lines 41-44).