If I train a GNB/LDA/kNN/other classifier I would like to know, in the model built, how important are features to classify or which feature(s) drives the classifier.
For example in SVM models the importance of the feature is sometimes evaluated looking at weights magnitude, but for non-linear and generative models is more difficult to extract weights.
Do you know a book/paper that could be useful to solve my question?
Best Answer
One famous paper is Wrappers for feature subset selection by Kohavi and John (1997). It basically deals with the problem of removing the irrelevant features, which should provide an answer to your question. The approach is agnostic wrt the classifier used.
It's old, and it has almost 4k citations, so there is probably a wealth of follow-up work.