Solved – Predict function tuning for random forest

rrandom forest

I have created a random forest object in R (using the randomForest package) with ntree = N. Now I would like to predict some new data on it using a subset of N, that is using only n trees for the prediction. Is this possible?

For the random forest object the forest is located at fit$forest, but I don't know how to extract them (if possible).

Best Answer

Sounds like you want to set predict.all = TRUE. This will cause predict.randomForest to return a list containing a vector of the aggregate predictions and a matrix of the individual tree predictions. You can then ensemble the individual trees at your leisure.

library("randomForest")
data(mtcars)

rf <- randomForest(mpg ~ ., data = mtcars, ntree = 10)
preds <- predict(rf, newdata = mtcars, predict.all = TRUE)

preds$aggregate   # Aggregate predictions
preds$individual  # Invididual tree predictions

Make sure you set newdata = <something> or this trigger fails for some reason.

Related Solutions

Solved – Does party package in R provide out-of-bag estimates of error for Random Forest models

The caret package has a method for getting that. You can use train as the interface. For example:

> mod1 <- train(Species ~ ., 
+               data = iris, 
+               method = "cforest", 
+               tuneGrid = data.frame(.mtry = 2),
+               trControl = trainControl(method = "oob"))
> mod1
150 samples
  4 predictors
  3 classes: 'setosa', 'versicolor', 'virginica' 

No pre-processing
Resampling: 

Summary of sample sizes:  

Resampling results

  Accuracy  Kappa
  0.967     0.95 

Tuning parameter 'mtry' was held constant at a value of 2

Alternatively, there is an internal function that can be used if you want to go straight to cforest but you have to call it using the namespace operator:

> mod2 <- cforest(Species ~ ., data = iris,
+                 controls = cforest_unbiased(mtry = 2))
> caret:::cforestStats(mod2)
 Accuracy     Kappa 
0.9666667 0.9500000

HTH,

Max

Solved – R – ROCR Library – Understanding predict and prediction method

Here are answers:

1) Pred is an intermediate object, from which one can can plot various metrics like AUC, ROC-curve, costs associated with various false classifications etc.

2) If you have estimated class probabilities for the pred object, plot can show classifications based on all cut-off points from 0..1, not just arbitrary cut-off like 0.5.

3) System cannot itself determine which arbitrary cut-off point for the class probability must be used. You have to decide yourself that after consulting various metrics. Are costs associated with false positive classification same as false negatives? If they are not equal, then you must adjust the cut-off point accordingly.

And perhaps you could just use raw class probabilities and select cases to analysis where ration of class probabilities to mean class probability is high enough AND costs/benefits warrant it?

Best Answer

Related Solutions

Solved – Does party package in R provide out-of-bag estimates of error for Random Forest models

Solved – R – ROCR Library – Understanding predict and prediction method

Related Question