I am trying to reach a stable model for my classification problem and I am using KNN for classification. In order to improve the accuracy I want to use bootstrap aggregation. In MATLAB I found a function "fitensemble". To my surprise in this function you cannot choose your learner as "KNN" and your ensemble learning method as "Bag". The only allowable ensemble learning method for KNN is "random subspace". I wonder if it is always like this and why? Am I missing something considering that I am quite new in this area.
Thanks in advance
Best Answer
You can use bootstrap aggregation (bagging) with any classifier! In fact, if you're crazy, you can bootstrap aggregate your bootstrap aggregates!
However, the author of the
fitensemble
function chose not to implement bagging for knn, which is too bad.I don't know much about matlab, but you could try the ipredknn functions from the ipred package in R, which will allow you to bag your knn models.