Solved – How to average multiple ROC curves in k-fold Cross-Validation

cross-validationmachine learningmeanrocstandard deviation

I am evaluating a Neural Network performance using 4-fold cross validation. I produced the ROC curve in the picture.

enter image description here

1) How can I average the folds to obtain just one ROC curve?

2) How can I justify this plot to retain it?
[ maybe it's useful to plot different folds to see the spread and how stable the model is, could you please explain this? ]

Best Answer

1) Calculate sensitivity and specificity at the incremental thresholds between 0 to 1 for all the folds. Averaging those should give you your desired average ROC Curve.

2)Displaying multiple plots can show you the spread but do not forget that randomly shuffling the same data can result in different spreads as well. The main reason to use cross-validation is mainly motivated by the fact that data spread is random. So try to use cross-validation as a validation tool for your model.