Solved – Decision Tree Interpretation (Classification using rpart)

analysisclassificationrrpart

When using rpart to create classification tree, the values for the relative importance of each predictor show up along these lines:

Var1: 33
Var2: 31
Var3: 25
Var4: 3

In my case Var3 is plotted as the root node. I expected that Var1 would have been the root node, given that it has the highest relative importance. Based on this, would it be reasonable to expect that Var1-3 would show up more and/or higher up towards the root of the tree? That question also applies to decision trees in general.

Thanks

Best Answer

Variable 3 is the predictor that provides the most separation in the two nodes after a single binary split. Predictors might show up multiple times further down the tree resulting greater overall importance.

Related Solutions

Solved – Finding the best combination of variables for high R-squared values

In your case, it might be feasible to try out all combinations (there are 16383 combinations) of sums. I wrote a quick and dirty implementation of that. With 14 variables it takes less than a minute to try out all combinations. If you want a random combination, you can modify the code to meet your needs.

my.vars <- matrix(NA, ncol=14, nrow=) # a matrix with your 14 different environmental variables
colnames(my.vars) <- paste("var", 1:14, sep="") # add row names "var1" - "var14"
my.grad.data <- 1:14
sum.vars <- vector()
r.2 <- vector()
comb.mat <- matrix(numeric(0), nrow=14, ncol=0) # initialise the matrix containing all combinations

for ( i in 1:14 ) { # generate and store all possible combination of sums of the 14 variables

  t.mat <- combn(my.grad.data, m=i)

  comb.mat <- cbind(comb.mat, rbind(t.mat, matrix(NA, ncol=dim(t.mat)[2] , nrow=14-i)))
}

for ( j in 1:dim(comb.mat)[2] ) { # calculate and store the R2 for all combinations

  sum.vec <- rowSums(my.vars[, comb.mat[, j]], na.rm=TRUE)

  sum.vars[j] <- paste(
    colnames(my.vars[, comb.mat[, j]])[!is.na(colnames(my.vars[, comb.mat[, j]]))], 
    collapse="+")

  r.2[j] <- summary(lm(allele ~ sum.vec))$r.squared 
}


result.frame <- data.frame(combination=sum.vars, r2=r.2)

result.frame.sorted <- result.frame[order(r.2, decreasing=TRUE), ]

head(result.frame.sorted, n=10) # the 10 "best" combinations

Solved – Regression tree algorithm with linear regression models in each leaf

While they work differently than your algorithm, I believe you'll find mob() and FTtree interesting. For Zeileis' mob see http://cran.r-project.org/web/packages/party/vignettes/MOB.pdf For FTtree,Gama's functional trees an implementation is available in Weka and thus RWeka. See http://cran.r-project.org/web/packages/RWeka/index.html for details.

Best Answer

Related Solutions

Solved – Finding the best combination of variables for high R-squared values

Solved – Regression tree algorithm with linear regression models in each leaf

Related Question