My question is about the 1-nearest neighbor classifier and is about a statement made in the excellent book The Elements of Statistical Learning, by Hastie, Tibshirani and Friedman. The statement is
"For k nearest neighbours the model complexity is controlled by k."
"Also as model complexity is increased low bias and high variance"
Now in KNN , if we have a smaller k, we have low bias and high variance.So does that mean a smaller k is a more complex model ?
This seems kind of counter-intuitive as I just chose on basis of 1 point. How is this model more complex?
Best Answer
I think it is right, 1NN is the most complex model as it will have the most number of effective parameters = N/k = N and the most convoluted boundaries