MATLAB: I have a confusion regarding getActionInfo and it not getting the right number of actions from the environment

environmentgetactioninforeinforcement learningReinforcement Learning Toolboxrlfinitesetspec

In my RL Environment Code Setup, I want to have two actions, action 1 and action 2 as follows:

ActionInfo = rlFiniteSetSpec([1 2]);
ActionInfo.Name = 'Action 1 or Action 2';

However, when I get action from my environment using the following code:

actInfo = getActionInfo(env)
numActions = actInfo.Dimension(2)

I get that numActions is 2. What should I fix?

P.S: This is what actInfo prints:


       Elements: [4×1 double]
           Name: "Action 1 or Action 2"
    Description: [0×0 string]
      Dimension: [1 1]
       DataType: "double"

Best Answer

Hi,

I could not reproduce the same at my end, as I get numActions as 1. Could you try the following commands, I suspect it might be due to any changes made at your end.

Try restarting your MATLAB and check again.

Also, what is the release version of MATLAB you are using.

rehash toolbox
rehash toolboxcache
which getActionInfo

Related Solutions

MATLAB: Specifying Actions using rlFiniteSetSpec or rlNumericSpec (Reinforcement Learning)

Hi Huzaifah,

It is my understanding that have 4 actions (U, D, L, R) and want to specify them in your RL agent.

Generally, rlNumericSpec is used when you have a continuous action space and rlFiniteSetSpec is used when you have a discrete action space.

In your case, the action space is discrete. So, you should go with rlFiniteSetSpec.

As rlFiniteSetSpec takes only numeric input, you have to encode your actions to numbers. For example, (U,D,L,R) -> (1,2,3,4)

>> actInfo = rlFiniteSetSpec([1,2,3,4])  
actInfo = 
  rlFiniteSetSpec with properties:
       Elements: [4×1 double]
           Name: [0×0 string]
    Description: [0×0 string]
      Dimension: [1 1]
       DataType: "double"

For more information you can refer to : https://in.mathworks.com/help/reinforcement-learning/ref/rl.util.rlfinitesetspec.html

MATLAB: How to extract a trained RL Agent’s network’s weights and biases

You can get the parameters from the trained's critic representation for DQN agent. In MATLAB R2020a, see getLearnableParameters and getCritic functions (function name changes a bit since R2019b). You can follow similar steps to get the actor's parameters from actor-based agent like DDPG or PPO.

critic = getCritic(agent);
criticParams = getLearnableParameters(critic);

Best Answer

Related Solutions

MATLAB: Specifying Actions using rlFiniteSetSpec or rlNumericSpec (Reinforcement Learning)

MATLAB: How to extract a trained RL Agent’s network’s weights and biases

Related Question