I'm using a Grid Search CV to find optimal hyperparameters for an SVM.
I want to use the best combination of hyperparameters and evaluate model performance using those hyperparameters. Can I just use the average accuracy given by the Grid Search CV as a metric for model performance?
I've looked at SE Post as well as SKLearn link, which indicates that the performance may be biased. However, why would the performance be biased? Isn't the performance metric given by the cross-validation for unseen data (namely, the held-out set in the cross-validation)? And as this unseen data is different for each fold, as well as the trained model, wouldn't the metric given back by the cross-validation be a good representation of how it would perform on unseen data?
Best Answer
Once you look at the CV results and choose a set of hyperparameters (into which all data involved in the CV in any fold enters), any data involved in that CV is not unseen any more to the final model.