As dpb said, it is impossible to know if some arbitrary value for RMSE is good or bad. Only you know if it is good, because only you can know how much noise you would expect in the data.
The point is, when you use a model on some data that generates an RMSE, there are TWO components to the error, noise and lack of fit.
Any model will not be able to predict random noise in the data, so the predictive capability of the model can be no better than that noise. If it is better, then you are overfitting the noise. This is a bad thing to do.
At the same time, a model is just a model. It is an approximation to the real thing, with some aspects of reality that are left out. In some cases, a model is just a metaphor for the real system under study, so your model arises from some completely different system, but one that you hope behaves similarly to the real world case under study. In either case, a model will typically have some degree of lack of fit, since it is never a perfect model of the process. There is always something you have not included.
So we think of the residuals for any model to a physical system as something composed of both noise and lack of fit. These two components mix together. If the model is a good one, and the lack of fit is very small, then all we will see after the model is removed is noise. So our hope is the RMSE will be roughly the same as the underlying noise standard deviation, something that you SHOULD have some feel for if you are modeling any process.
So if the RMSE is significantly very small, more so than we would expect, then you are overfitting the problem. That is bad, since your model is now compensating for (chasing after) random noise.
If the RMSE is significantly larger than you would have expected, then this is again a problem, since it indicates there is a potential problem in the model. The model is inadequate to represent the system under study, or you did a poor job of estimating anything that must be estimated in the model.
But the point is, RMSE means NOTHING unless you have some a-priori knowledge about the process. You must have some expectation as to what is acceptable for the problem you are studying. IF you have no clue, it is impossible for someone else to know, since did not develop the model, we did not generate the data under study.
Best Answer