MATLAB: Remove or replace trees from a TreeBagger ensemble

ensembleMATLABregressionStatistics and Machine Learning Toolboxtreebagger

Hello,
There is this method TreeBagger.combine that can combine two independently trained TreeBagger classifiers.
The problem is that I have to combine two ensembles, but I want to replace randomly 25% of the trees from the first ensemble with trees from the second ensemble.
Three possible solutions to overcome my problem:
1. Method to remove specified trees from ensemble. Then I would just remove 25% of the ensemble with random indexes and combine with the second ensemble.
2. Method to construct new TreeBagger from a collection of trees. For example something like this:
       TreeBagger=Construct(Tree1, Tree2, ...);
3. Method to replace trees in TreeBagger ensemble. For example I tried to do this, like in a structure:
       ensemble.Trees{i}=new_tree;
but that didn't work, because the property is private.
Please help me, because I had no progress with any of three…

Best Answer

Combining two objects would be hard. You can work around this by growing one big ensemble and treating parts of it as separate ensembles. Take a look at the 'trees' argument for PREDICT and similar mehods.
This recipe is justified because every tree in a TreeBagger ensemble is grown independently of others. For prediction, trees are combined by simple averaging. This approach would not work for ensembles of other types.
Here is how I effectively grow two TreeBagger ensembles and then combine them:
% Load data
load ionosphere;
% Grow two ensembles, 100 trees each
b = TreeBagger(200,X,Y,'oobpred','on');
% Out-of-bag prediction from 1st ensemble
Yoob1 = oobPredict(b,'trees',1:100);
% Out-of-bag prediction from 2nd ensemble
Yoob2 = oobPredict(b,'trees',101:200);
% Randomly drop 25 trees from the 1st ensemble and combine the two
% ensembles.
idx1 = datasample(1:100,75,'replace',false);
Yoob3 = oobPredict(b,'trees',[idx1 101:200]);
Related Question