At the start of every game, a random number of no-op actions are played (with the maximum number of such actions controlled by that parameter) to introduce variety in the initial game states.
If an agent starts from exactly the same initial state every time it plays the same game, they're afraid that the Reinforcement Learning will simply learn to memorize a good sequence of actions from that initial state, rather than learning to observe the current state and select a good action based on that observation (which is the more interesting thing we're interested in learning). The idea is that by introducing randomness in the state we "start playing from", it should become impossible / more difficult for the agent to "cheat" and simply memorize a complete sequence of actions from a single specific initial state.
Note that, in 2017, it has been argued in the Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents paper that these kinds of sequences no-op actions are not as effective at the goal described above as we would like, and an alternative solution is proposed which consists of introducing stochasticity throughout the entire game through "sticky actions" (basically sometimes randomly continue with the most recently-selected action rather than a new action selected by the agent).
Perhaps this should be rephrased as "attribution", but in many RL models, the signal that comprises the reinforcement (e.g. the error in the reward prediction for TD) does not assign any single action "credit" for that reward. Was it the right context, but wrong decision? Or the wrong context, but correct decision? Which specific action in a temporal sequence was the right one?
Similarly, in NN, where you have hidden layers, the output does not specify what node or pixel or element or layer or operation improved the model, so you don't necessarily know what needs tuning -- for example, the detectors (pooling & reshaping, activation, etc.) or the weight assignment (part of back propagation). This is distinct from many supervised learning methods, especially tree-based methods, where each decision tells you exactly what lift was given to the distribution segregation (in classification, for example). Part of understanding the credit problem is explored in "explainable AI", where we are breaking down all of the outputs to determine how the final decision was made. This is by either logging and reviewing at various stages (tensorboard, loss function tracking, weight visualizations, layer unrolling, etc.), or by comparing/reducing to other methods (ODEs, Bayesian, GLRM, etc.).
If this is the type of answer you're looking for, comment and I'll wrangle up some references.
Best Answer
In the paper you mention, they seem to be more flexible regarding the meaning of epoch, as they just define one epoch as being a certain amount of weight updates. You can therefore view one epoch as being an outer loop around the episode loop, as you mentioned in the question.