I'm trying to build a recommendation system, and have a bunch of (item,item_features,liked) triplets, where liked is binary. Most items are not liked. So I'm running a logistic regression with glmnet of the form
liked ~ item_features
This yields an AUC of around 0.75 (it doesn't vary much with the regularization parameter). However, the error rate (also doesn't vary much) is only a tiny,tiny bit better than what you would get if you just always predicted "don't like." What is the best way (or any way, really!) to think about the value or lack thereof this recommender?
Solved – AUC vs error rate for classification
aucclassificationinterpretationrecommender-system
Best Answer
0.75 is a pretty modest discrimination, I would say that's a decent model in the case of recommender systems where the volume of material someone is exposured to is gigantic relative to what the encounter and eventually like. You'd expect only an incremental positive predictive value in that case, since the outcome is rare, so discounting the model for having low PPV would be too harsh.