Solved – Complexity parameter zero in decision trees

cart

As I understand from this reference (cran.r-project.org/web/packages/rpart/vignettes/longintro.pdf , pp. 12-13) about the rpart package , the criteria for a making (or not) a new split in a decision tree is to compare the decrease in the error of the tree with the new split against the complexity parameter times the number of leaves it would yield. If the former is greater, then the split is made.

I've grown a tree of cp=0 for the whole iris dataset, and the tree I get is the following:

enter image description here

The predictor space is split as follows:

enter image description here

So my question is: if the complexity parameter is zero, why does the splitting end there? Why doesn't the algorithm split the versicolor node further with a Petal.Length<5.4? This would decrease the error of the tree, and since the complexity parameter is zero, it wouldn't prevent it from happening… or not??

Best Answer

Short answer: If you set minsplit=1 you may get what you want.

Long answer: CART build tree in 2 stages.

  • The first stage will grow tree as much as possible (with specified constrains)
  • The second stage will prune the tree use 1 SE rule

There are may parameters to control how to grow the tree and how to prune the tree, setting cp=0 is equal to say do not prune the tree. But we may need to set maxdepth, minsplit, minbuket etc. to control how to grow the tree.

This is my question and answer to the same question, with more complicated data than iris.

Why I cannot achieve 100% accuracy in my simple training data with CART model?

Related Question