Solved – Root MSE or (RMSE) in regression model

mseregressionrms

I was calculating the root mean squared error for a regressive model, and while calculating, its mean squared error (MSE) came out to be less than 1 (>0), and its root (the root mean squared error, RMSE) as obvious came out to be greater than it. What does that mean?
like MSE = 0.0929285270191
and hence, RMSE = 0.894951677992

How do I see it?
Was there any problem in calculation or training of the model?
Is it that MSE is coming out very less that is not likely possible?

Best Answer

Your calculation must be wrong somewhere.

As the name says the RMSE is the square root of the MSE, so for MSE = 0.09 the RMSE would be 0.3. Yours is approximately off by a factor of 3.

Related Solutions

Solved – the “root MSE” in Stata

Calculate the difference between the observed and predicted dependent variables
Square them
Add them up, this will give you the "Error sum of squares," SS in Stata output
Divide it by the error's degrees of freedom, this will give you the "Mean error sum of squares," MS in Stata output
Take a square root of it, and this is the Root MSE
Done

If you look at the Stata output:

. sysuse auto, clear
(1978 Automobile Data)

. reg mpg weight

      Source |       SS       df       MS              Number of obs =      74
-------------+------------------------------           F(  1,    72) =  134.62
       Model |   1591.9902     1   1591.9902           Prob > F      =  0.0000
    Residual |  851.469256    72  11.8259619           R-squared     =  0.6515
-------------+------------------------------           Adj R-squared =  0.6467
       Total |  2443.45946    73  33.4720474           Root MSE      =  3.4389

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      weight |  -.0060087   .0005179   -11.60   0.000    -.0070411   -.0049763
       _cons |   39.44028   1.614003    24.44   0.000     36.22283    42.65774
------------------------------------------------------------------------------

Dividing the sum of squares of the residual (851.469) by its degrees of freedom (72) yields 11.826. That is the mean sum of squares. If you further take a square root, you'll get Root MSE (3.4289 in the output).

Basically, it's a measurement of accuracy. The more accurate model would have less error, leading to a smaller error sum of squares, then MS, then Root MSE. However, you can only apply this comparison within the same dependent variables, because MS and Root MSE are not standardized. Depending on the unit of measurements, Root MSE can vary greatly.

Mean Absolute Error – MAE vs Root Mean Squared Error in Regression Analysis

This depends on your loss function. In many circumstances it makes sense to give more weight to points further away from the mean--that is, being off by 10 is more than twice as bad as being off by 5. In such cases RMSE is a more appropriate measure of error.

If being off by ten is just twice as bad as being off by 5, then MAE is more appropriate.

In any case, it doesn't make sense to compare RMSE and MAE to each other as you do in your second-to-last sentence ("MAE gives a lower error than RMSE"). MAE will never be higher than RMSE because of the way they are calculated. They only make sense in comparison to the same measure of error: you can compare RMSE for Method 1 to RMSE for Method 2, or MAE for Method 1 to MAE for Method 2, but you can't say MAE is better than RMSE for Method 1 because it's smaller.

Best Answer

Related Solutions

Solved – the “root MSE” in Stata

Mean Absolute Error – MAE vs Root Mean Squared Error in Regression Analysis

Related Question