Solved – How to test if two regression models are significantly different in R

ancovarregressionstatistical significance

Generally speaking I want to find out if the regression models I built are significantly different from each other. I have monthly energy consumption data (the independent variable) from a couple of houses and Heating Degree Days (HDD) covering a period of about 3 years. The data has been divided into groups with Heating Season and House. For each house I have 3 (three heating seasons) regression models, and I would like to test if they are significantly different from each other. Maybe to make it less complicated I would like to compare model of heating season HS 1 with HS 2, and HS 2 with HS 3.

On different forums I have read that I can use ANCOVA to compare models but not being advanced in statistics, I do not really know how to do it in practice. But I made myself familiar with R to a degree so I am trying to solve my problem with this software.
I would appreciate your suggestions how to sort out this issue using R.

Here is an extract of data for one house.

HDD    Energy   HeatingSeason    House
17.5       18            HS 1        1
84.4    206.5            HS 1        1
218.2   677.7            HS 1        1
374.4   824.4            HS 1        1
263.47  602.4            HS 1        1
174.27  514.4            HS 1        1
188.75  436.1            HS 1        1
58.59    44.3            HS 1        1
45.23       0            HS 2        1
26.14       0            HS 2        1
6.03        0            HS 2        1
6.95        0            HS 2        1
13.77       0            HS 2        1
51.82    45.1            HS 2        1
100.57  320.6            HS 2        1
208.78  634.5            HS 2        1
216.02  692.8            HS 2        1
275.34  643.2            HS 2        1
157.03  197.1            HS 2        1
165.42  246.7            HS 2        1
73.06    69.3            HS 3        1
46.21     3.9            HS 3        1
1.11        0            HS 3        1
17.15       0            HS 3        1
60.98    28.7            HS 3        1
164.28  197.1            HS 3        1
225.94  456.8            HS 3        1
315.3   687.8            HS 3        1
407.82  800.5            HS 3        1
342.88  771.2            HS 3        1
346.14  645.1            HS 3        1
191.03  223.7            HS 3        1

Best Answer

Yes you can only use ANOVA for nested models. That means you can compare models of the form Energy ~ HDD + House and Energy ~ HDD, but not across the same model used with different samples. Instead, take a look at the response to this question: Test equivalence of non-nested models.

Taken from that thread, you can use some different functions:

library(lmtest)
coxtest(fit1, fit2)
jtest(fit1, fit2)

Coxtest is the Cox LR test, and jtest is the Davidson-MacKinnon J test.

Related Question