MATLAB: How to have several actions for a DQN in the Reinforcement Learning Toolbox

actionscustom environmentreinforcement learningrlfinitesetspec

I'm trying to define the output of a DQN agent with a custom environment, and can't use the actionInfo = rlFiniteSetSpec() correctly.

I'm trying to control 3 actuators that will receive commands 0 and 1.

I always get number of actions = 1.

And the documentation is not clear as it's a new toolbox.

Any suggestions?

Best Answer

If you type

help rlFiniteSetSpec

the second example is

spec = rlFiniteSetSpec({[0,1];[1,1];[1,2];[1,3]})

If you define all possible combinations of the discrete inputs in a cell array as above, that should work (think of a single action as one possible combination of your 3 actuator commands).

I hope this helps.

Related Solutions

MATLAB: How to extract a trained RL Agent’s network’s weights and biases

You can get the parameters from the trained's critic representation for DQN agent. In MATLAB R2020a, see getLearnableParameters and getCritic functions (function name changes a bit since R2019b). You can follow similar steps to get the actor's parameters from actor-based agent like DDPG or PPO.

critic = getCritic(agent);
criticParams = getLearnableParameters(critic);

MATLAB: Specifying Actions using rlFiniteSetSpec or rlNumericSpec (Reinforcement Learning)

Hi Huzaifah,

It is my understanding that have 4 actions (U, D, L, R) and want to specify them in your RL agent.

Generally, rlNumericSpec is used when you have a continuous action space and rlFiniteSetSpec is used when you have a discrete action space.

In your case, the action space is discrete. So, you should go with rlFiniteSetSpec.

As rlFiniteSetSpec takes only numeric input, you have to encode your actions to numbers. For example, (U,D,L,R) -> (1,2,3,4)

>> actInfo = rlFiniteSetSpec([1,2,3,4])  
actInfo = 
  rlFiniteSetSpec with properties:
       Elements: [4×1 double]
           Name: [0×0 string]
    Description: [0×0 string]
      Dimension: [1 1]
       DataType: "double"

For more information you can refer to : https://in.mathworks.com/help/reinforcement-learning/ref/rl.util.rlfinitesetspec.html

Best Answer

Related Solutions

MATLAB: How to extract a trained RL Agent’s network’s weights and biases

MATLAB: Specifying Actions using rlFiniteSetSpec or rlNumericSpec (Reinforcement Learning)

Related Question