I was watching a very interesting video with Yoshua Bengio where he is brainstorming with his students. In this video they seem to make a distinction between "credit assignment" vs gradient descent vs back-propagation. From the conversation it seems that the credit assignment problem is associated with "backprop" rather than gradient descent. I was trying to understand why that happened. Perhaps what would be helpful was if there was a very clear definition of "credit assignment" (specially in the context of Deep Learning and Neural Networks).
What is "the credit assignment problem:?
and how is it related to training/learning and optimization in Machine Learning (specially in Deep Learning).
From the discussion I would have defined as:
The function that computes the value(s) used to update the weights. How this value is used is the training algorithm but the credit assignment is the function that processes the weights (and perhaps something else) to that will later be used to update the weights.
That is how I currently understand it but to my surprise I couldn't really find a clear definition on the internet. This more precise definition might be possible to be extracted from the various sources I found online:
-
https://www.youtube.com/watch?v=g9V-MHxSCcsWhat is the "credit assignment" problem in Machine Learning and Deep Learning?
-
How Auto-Encoders Could Provide Credit
Assignment in Deep Networks via Target Propagation https://arxiv.org/pdf/1407.7906.pdf -
Yoshua Bengio – Credit assignment: beyond backpropagation (https://www.youtube.com/watch?v=z1GkvjCP7XA)
-
Learning to solve the credit assignment problem (https://arxiv.org/pdf/1906.00889.pdf)
Also, I found according to a Quora question that its particular to reinforcement learning (RL). From listening to the talks from Yoshua Bengio it seems that is false. Can someone clarify how it differs specifically from the RL case?
Cross-posted:
Best Answer
Perhaps this should be rephrased as "attribution", but in many RL models, the signal that comprises the reinforcement (e.g. the error in the reward prediction for TD) does not assign any single action "credit" for that reward. Was it the right context, but wrong decision? Or the wrong context, but correct decision? Which specific action in a temporal sequence was the right one?
Similarly, in NN, where you have hidden layers, the output does not specify what node or pixel or element or layer or operation improved the model, so you don't necessarily know what needs tuning -- for example, the detectors (pooling & reshaping, activation, etc.) or the weight assignment (part of back propagation). This is distinct from many supervised learning methods, especially tree-based methods, where each decision tells you exactly what lift was given to the distribution segregation (in classification, for example). Part of understanding the credit problem is explored in "explainable AI", where we are breaking down all of the outputs to determine how the final decision was made. This is by either logging and reviewing at various stages (tensorboard, loss function tracking, weight visualizations, layer unrolling, etc.), or by comparing/reducing to other methods (ODEs, Bayesian, GLRM, etc.).
If this is the type of answer you're looking for, comment and I'll wrangle up some references.