I have implemented linear regression manually, for learning purposes, and I use the "Auto MPG" data set as the toy data I'm applying it to.
It occured to me that I don't know how to test the efficiency of my model! With classification I can check the predicted class vs the actual class, but what do I do with regression?
EDIT: Of course, I am splitting the data I have into train + test data…I am speaking of evaluating the results when I apply the model on the test data.
For instance, using my model I predict 16.76 mpg for a car for which the actual value in the data set is 15.5. How do I decide whether this is a "good" or "bad" prediction?
I am thinking of using some thresholds ("if the predicted value is within the [actual-epsilon, actual+epsilon] interval => OK!"), but is this a good approach? And even if it is, how do I choose epsilon values?
I am aware of the fact that there's most likely no clear-cut answer, but any suggestion regarding the approach to take would be most helpful.
EDIT: I've decided to go with the following approach: for each test data sample, I am computing how far the predicted value is from the test data as
abs(predicted_value - actual_value) * 100 / actual_value
i.e. how far away is the prediction from the actual value, expressed in % of actual value.
Any feedback is still welcome, since this is just an idea I've had, and I'm not sure if it is "best practice".
Best Answer
I suggest using RMSE (root mean square error) of your predictions on your test set when compared to the actual value. This is a standard method of reporting prediction error of a continuous variable. You can use R^2 to examine how well your model fits the training data.