MATLAB: Unable to run ‘rlwatertank’ example in R2020a

Question

Hello everyone

I was trying to run this example.

While I successfully ran this example in R2019b, I could not successfully train the agent for this example in R2020a.

I also tried other available examples in the documentation; however, the learning plots in Reinforcement Learning Episode Manager differed from the plots exhibited in documentation.

I should mention I followed the exact steps in documentation and did not change value of any parameter.

Is this some sort of bug in "Reinforcement Learning Toolbox" at R2020a release?

Answer 1

Hi Nima,

This is the plot I got when running the watertank example in 20a:

While this is not exactly the same as the one shown in the documentation, training still converges.

A couple of reasons why the visual is not the same as in R2019b:

1. We recently started using autodifferentiation under the hood, and while gradient values are still close, there are small numerical differences which lead to a different optimization route,
2. Each release comes with more optimization improvements like the one above, which affect training results.

Note that training should still converge. Hope this helps