I understand theoretically (sort of) how they would work, but am not sure how to go about actually making use an ensemble method (such as voting, weighted mixtures, etc.).
- What are good resources for implementing ensemble methods?
- Are there any particular resources regarding implementation in Python?
EDIT:
To clear up some based on the discussion on the comments, I'm not looking for ensemble algorithms such as randomForest, etc. Instead, I'm wondering how can you combine different classifications from different algorithms.
For example, say someone uses logistic regression, SVM, and some other methods to predict the class of a certain observation. What is the best way to go about capturing the best estimate of the class based upon these predictions?
Best Answer
A good place to start is to get an overview of ensemble learning. Especially you'll want to look at boosting and bagging. Another method was that used by "The Ensemble" team in the Netflix Prize, is called either "blending" or feature stacking.
Then, just find some libraries that implement those and work from there. A quick googling turned up scikit and orange, both of which should have bagging and boosting (and they're both Python).
If beyond just using ensemble methods, you'd like to learn a bit of the theory, then I think this paper would be a good jumping off point (follow the references for the parts you're interested in).
Cheers.