MATLAB: Deep reinforcement learning for multi-agents

deep reinforcement learningmulti-agents

By the multi-agent deep reinforcement learning toolbox, three agents are trained. The reward changes are as shown in the picture. Why do agents' rewards decrease and converge to an unfavorable situation after the reward increases and they move towards desired performance? I expected the process of increasing the rewards and achieving the desired goal to continue as the episode progresses. According to the picture, from episode 700, agents converge to undesired situations, and they didn't change their states.

Thank you.

Best Answer

Hello,

The policies you will get from RL training change depending on the amount of time the agents spend exploring. Usually, if you see a situation like this where agents converge to a non-ideal solution, you may want to change the agent options to increase exploration.

Hope that helps

Related Solutions

MATLAB: Multi-agent deep reinforcement learning

Assuming you are training multiple agents in Simulink using the Reinforcement Learning Toolbox in R2020b:

The rewards are calculated by the environment, not the agent algorithm so they should not be affected unless the environment is changing them. When you compare rewards between single and multi-agents please ensure that the state-action pairs are the same. Rewards depend on states and actions and you may get different results for different state-action pairs.
In R2020b, the agent neural networks are updated independently.

MATLAB: How to extract a trained RL Agent’s network’s weights and biases

You can get the parameters from the trained's critic representation for DQN agent. In MATLAB R2020a, see getLearnableParameters and getCritic functions (function name changes a bit since R2019b). You can follow similar steps to get the actor's parameters from actor-based agent like DDPG or PPO.

critic = getCritic(agent);
criticParams = getLearnableParameters(critic);

Best Answer

Related Solutions

MATLAB: Multi-agent deep reinforcement learning

MATLAB: How to extract a trained RL Agent’s network’s weights and biases

Related Question