I am asking myself if it is a good idea to remove those variables with a negative variable importance value ("%IncMSE") in a regression context. And if it gives me a better prediction? What do you think?
Solved – Variable importance randomForest negative values
feature selectionimportancemachine learningrandom forest
Best Answer
Variable importance in Random forest is calculated as follows:
Then, the values of a single column are permuted and the MSE is calculated again. For example, If a column (Col1) takes the values 1,2,3,4, and a random permutation of the values results in 4,3,1,2. This results in an MSE1. Then an increase in the MSE, i.e., MSE1 - MSE, would signify the importance of the variable.
We expect the difference to be positive, but in the cases of a negative number, it denotes that the random permutation worked better. It can be inferred that the variable does not have a role in the prediction,i.e, not important.
Hope this helps!
Please refer to the following link for a elaborated explanation!
https://stackoverflow.com/questions/27918320/what-does-negative-incmse-in-randomforest-package-mean