I'm studying Random Forests, but after reviewing the methods I got the following line of reasoning: I feel like the big advantage of random forests happens in the bagging process where nearly uncorrelated predictions are created due to the random features, producing predictions with low variance. On the other hand, this method doesn't require your classification method to be a decision tree.
If other classification algorithms (LDA, QDA, logistic regression, etc) usually perform better than decision trees, why would I choose to use decision trees as my classifier?
Random forests seem a lot more popular than bagging with other classifiers (it even got its own name) but I don't see any particular reason why.
Best Answer
I find this a great question. As mentioned by @guy in the comments section, bagging is more useful when the estimator / training algorithm is highly unstable and, I would add to that, non-linear. OLS or MLE are exact methods, on the same dataset they will always converge to the same solution.
CART on the other hand solves an NP-hard problem via greedy search and will produce different results even on the same dataset. There is a significant added overfitting/variance in CART's predictions and this is what bagging makes use of.