Solved – What do interactions of spline and non-spline terms mean

interactionmultiple regressionrsplines

If I fit my data with something like lm(y~a*b), in R syntax, where a is a binary variable and b is a numeric variable, then the a:b interaction term is the difference between slope of y~b at a = 0 and at a = 1.

Now, let's say the relationship between y and b is curvilinear. If I now fit lm(y~a*poly(b,2)), then a:poly(b,2)1 is the change in the change in y~b conditional on the level of a as above, and a:poly(b,2)2 is the change in y~b^2 conditional on the level of a. It takes some handwaving, but if either of those interaction coefficients are significantly different from zero, I can argue that it means a affects not only the vertical displacement of y but also the location of the peak and the steepness of approach to the peak of the y~b+b^2 curve.

What about if I fit lm(y~a*bs(b,df=3))? How do I interpret the a:bs(b,df=3)1, a:bs(b,df=3)2, and a:bs(b,df=3)3 terms? Are these the vertical displacements of y from the spline attributable to a at each of the three segments?

Best Answer

+1 for a good, and clearly-stated question. (If you wanted a little more information about polynomials and splines, you might find this helpful, although you seem to have a strong grasp of the topic.) You may also want to read this recent question regarding the interpretation of terms governing the curvature of the relationship between a covariate and a response variable. You will notice that I argue against giving separate interpretations to the different terms, but that it is best to treat them as gestalts. (However not to take too hard a line, I do recognize that you can calculate the location of the apex of the parabola from the betas of the regression model as you note here.) Consistent with my previous answer, I think it is best to interpret all the terms associated with the same underlying variable together. With respect to this specific case, the interaction simply establishes that the shape of the curves differs between the two levels of factor a.

Related Solutions

Solved – Interactions between non-linear predictors

You could try generalized additive mixed models, handily implemented in the gamm4 package. The way I've used them, you can do something like:

fit1 = gamm4(
    formula = V1 ~ V2 + s(V3)
    , random = ~ (1|V4)
)
fit2 = gamm4(
    formula = V1 ~ V2 + s(V3,by=V2)
    , random = ~ (1|V4)
)

fit1 seeks to predict V1 using V4 as a random effect and V2 and V3 as fixed effects, but where V3 is spline-smoothed. fit2 seeks the same, except with the addition of permitting the smooth of V3 to vary within levels of V2, thus implementing an interaction. Comparison of fit1 to fit2 evaluates the necessity of permitting the interaction.

Solved – Feature standardization for polynomial regression with categorical data

When a LASSO model includes a categorical predictor with more than 2 levels, you usually want to ensure that all levels of the predictor are selected together as with the group LASSO. When a LASSO model includes interaction terms, it's important to maintain the hierarchy of the interactions. That is, if LASSO selects an interaction term it should also select the terms of the individual predictors contributing to the interaction. That's discussed briefly here and with more rigor by Bien, Taylor and Tibshirani in "A lasso for hierarchical interactions", Ann. Stat. 41; 1111–1141, 2013.

For your questions 1 and 3, Bien, Taylor and Tibshirani seem to deal directly with your question:

It is common with the lasso to standardize the predictors so that they are on the same scale. In this paper, we standardize X [matrix of individual predictors] so that its columns have mean 0 and standard deviation 1; we then form Z [matrix of interaction terms] from these standardized predictors and, finally, center the resulting columns of Z.

As the quadratic terms in your model are essentially self-interactions it would seem that you would be advised to proceed similarly. That is, standardize the continuous predictors $x_1$ and $x_2$ (subtract mean, divide by standard deviation), form the polynomial and interaction terms from the standardized predictors, then only center the polynomial and interaction terms. (As I understand it the centering of the interactions isn't necessary but does simplify interpretations of coefficients.) The corresponding R hierNet package by Bien and Tibshirani provides those choices as defaults: center features, standardize main effects, and don't standardize interactions. The hierNet() function does allow for other choices, if you want to play with other possibilities.

With respect to question 2, as noted in a comment it's not clear whether or how best to standardize a categorical predictor, particularly with more than 2 levels. Provided you handle it with group LASSO and respect the hierarchy of interactions, however, there isn't any problem in "deal[ing] with the interaction terms of categorical and continuous variables." If you choose treatment coding of the categorical predictor then the coefficients of the continuous predictors and their interactions with each other represent those values when the categorical predictor is at its reference level. The corresponding interaction terms with the other levels of the predictor are the differences of the coefficients from those values for the reference level. I see nothing to be gained by incorporating powers of the dummy variables representing the categorical predictor.

With respect to question 4, the "alternating signs" in interaction values after centering are features, not bugs. See this page for example. Leave them alone.

Best Answer

Related Solutions

Solved – Interactions between non-linear predictors

Solved – Feature standardization for polynomial regression with categorical data

Related Question