Solved – importance of predictor variables in multiple linear regression

multiple regressionr

I am running multiple linear regression with R.

mod=lm(varP ~ var1 +var2+var3+var4)

The table is:

all:
lm(formula = varP ~ var1 + var2 + var3 + var4)

Residuals:
    Min      1Q  Median      3Q     Max     
-4.9262 -0.6985  0.0472  0.7319  4.3305 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.700823   0.084737   8.271 1.45e-15 ***
var1      1.080172   0.175348   6.160 1.59e-09 ***
var2     -0.057803   0.007777  -7.432 5.25e-13 ***
var3     -9.924772   4.268235  -2.325   0.0205 *  
var4     -0.015104   0.001290 -11.710  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 1.139 on 460 degrees of freedom
Multiple R-squared:  0.657, Adjusted R-squared:  0.654 
F-statistic: 220.3 on 4 and 460 DF,  p-value: < 2.2e-16

it means that my model explains 65.4% of the variance.
But now, I would like to determine the importance of each predictor.

I was using:

lm.sumSquares(mod)

Is dR-sqr relevant to interpret this importance ?

              SS       dR-sqr pEta-sqr  df        F p-value
(Intercept)   88.73054 0.0510   0.1294   1  68.4015  0.0000
var4         177.88026 0.1022   0.2296   1 137.1262  0.0000
var2          71.65234 0.0412   0.1072   1  55.2361  0.0000
var1          49.22579 0.0283   0.0762   1  37.9477  0.0000
var3           7.01377 0.0040   0.0116   1   5.4069  0.0205

Error (SSE)  596.71237     NA       NA 460       NA      NA    
Total (SST) 1739.76088     NA       NA  NA       NA      NA

Best Answer

If you are using R you can use the caret package which has a built in method to give variable importance. See this link (http://caret.r-forge.r-project.org/varimp.html)

You basically will just have to do

 varImp(mod, scale = FALSE)

Related Solutions

Solved – When to transform predictor variables when doing multiple regression

I take your question to be: how do you detect when the conditions that make transformations appropriate exist, rather than what the logical conditions are. It's always nice to bookend data analyses with exploration, especially graphical data exploration. (Various tests can be conducted, but I'll focus on graphical EDA here.)

Kernel density plots are better than histograms for an initial overview of each variable's univariate distribution. With multiple variables, a scatterplot matrix can be handy. Lowess is also always advisable at the start. This will give you a quick and dirty look at whether the relationships are approximately linear. John Fox's car package usefully combines these:

library(car)
scatterplot.matrix(data)

Be sure to have your variables as columns. If you have many variables, the individual plots can be small. Maximize the plot window and the scatterplots should be big enough to pick out the plots you want to examine individually, and then make single plots. E.g.,

windows()
plot(density(X[,3]))
rug(x[,3])
windows()
plot(x[,3], y)
lines(lowess(y~X[,3]))

After fitting a multiple regression model, you should still plot and check your data, just as with simple linear regression. QQ plots for residuals are just as necessary, and you could do a scatterplot matrix of your residuals against your predictors, following a similar procedure as before.

windows()
qq.plot(model$residuals)
windows()
scatterplot.matrix(cbind(model$residuals,X))

If anything looks suspicious, plot it individually and add abline(h=0), as a visual guide. If you have an interaction, you can create an X[,1]*X[,2] variable, and examine the residuals against that. Likewise, you can make a scatterplot of residuals vs. X[,3]^2, etc. Other types of plots than residuals vs. x that you like can be done similarly. Bear in mind that these are all ignoring the other x dimensions that aren't being plotted. If your data are grouped (i.e. from an experiment), you can make partial plots instead of / in addition to marginal plots.

Hope that helps.

Solved – Categorical predictor variables in multiple regression

Can you? Sure. But it'll no doubt be less work to use ANOVA. Mathematically, ANOVA is equivalent to regression; they're versions of the same general linear model. The mechanics vary depending on the software you're using, but ordering up an interaction (to show effects associated with combinations of predictors) is simpler if you use, e.g., Age*Academic Discipline in ANOVA than if you have to account for the many dummy variables you've created for regression.

Related Question