I'm wondering if I should check if a Matrix Factorization model I built for recommendation by Collaborative Filtering is overfitting.
I trained a model using MLlib ALS (Alternating Least Squares) method and it works fine for the testing data set, but on training data it was predicting almost perfectly, which is logical since I put those number into the matrix.
Is there another way to check for overfitting? Or do I not need to worry about overfitting on Matrix Factorization models?
Best Answer
The most basic thing to do if you suspect overfitting is to plot the leaning curves for the training and the test set. So train your model on a small sample, then bigger, bigger, until its the full training set. Plot error for all the test set and the subset of examples you used for training.
These curves will show if your model starts overfitting and when, also you will get a clue on the bias-variance state of your model.