Solved – Can Bootstrap aggregation be used with knn classifier

aggregationbootstrapk nearest neighbourMATLAB

I am trying to reach a stable model for my classification problem and I am using KNN for classification. In order to improve the accuracy I want to use bootstrap aggregation. In MATLAB I found a function "fitensemble". To my surprise in this function you cannot choose your learner as "KNN" and your ensemble learning method as "Bag". The only allowable ensemble learning method for KNN is "random subspace". I wonder if it is always like this and why? Am I missing something considering that I am quite new in this area.

Thanks in advance

Best Answer

You can use bootstrap aggregation (bagging) with any classifier! In fact, if you're crazy, you can bootstrap aggregate your bootstrap aggregates!

However, the author of the fitensemble function chose not to implement bagging for knn, which is too bad.

I don't know much about matlab, but you could try the ipredknn functions from the ipred package in R, which will allow you to bag your knn models.

Related Question