MATLAB: Reinforcement learning toolbox – q table

q-learningreinforcement learning

I'm a newbie to RL and the RL toolbox. I played with Q-learning agent with a model in simulink. My question is after training, How can I access to the trained Q table? The qTable used to generate the agent is all ZERO. I cannot figure out where the trained Q values and the policies are stored. Thank you!

Best Answer

Hi Xinpeng,

To see the trained table, you have to do is extract it using ‘getCritic’. Try:

critic = getCritic(agent);

The variable ‘critic’ has a field which contains the Qtable after training.

Related Solutions

MATLAB: How to extract a trained RL Agent’s network’s weights and biases

You can get the parameters from the trained's critic representation for DQN agent. In MATLAB R2020a, see getLearnableParameters and getCritic functions (function name changes a bit since R2019b). You can follow similar steps to get the actor's parameters from actor-based agent like DDPG or PPO.

critic = getCritic(agent);
criticParams = getLearnableParameters(critic);

MATLAB: QTable reset when using train

If you stop training, you should be able to continue from where you left off. I called 'train' on the basic grid world example a couple of times in a row and the output of 'getLearnableParameters(getCritic(qAgent))' was different. You can always save the trained agent and reload it as well to make sure you don't accidentally delete it.

Update:

There is a regularization term added to the loss which causes the other entries to change slightly. To avoid this, you can type:

qRepresentation.Options.L2RegularizationFactor=0;

Best Answer

Related Solutions

MATLAB: How to extract a trained RL Agent’s network’s weights and biases

MATLAB: QTable reset when using train

Related Question