I am using scikit learn to build a Random Forest classifier. I have heard that it might be possible to build a single decision tree from a Random Forest. The suggestion is that although the decision tree may not be as good a classifier as the Random Forest it may be better than the decision tree you would get by using the standard method.
However I haven't been able to find this method online. Does it exist?
My question is not about extracting one of the decision trees from a Random Forest. It is asking about a method to construct a new decision tree from the whole Random Forest, perhaps by combining the trees in the Random Forest somehow.
Best Answer
Perhaps what you are looking for is the Combining Multiple Models (CMM) approach developed by Domingos in the 90s. Details for using this with a bagged ensemble of C4.5 rules is described in his ICML paper
The pseudocode in Table 1 is not specific to bagged C4.5, however:
To apply this to Random Forests, the key issue seems to be how to generate the randomly generated example $\overrightarrow{x}$. Here is a notebook showing one way to do it with
sklearn.This has got me wondering what follow-up work has been doing on CMM, and if anyone has come up with a better way to generate $\overrightarrow{x}$. I've created a new questions about it here.