we use models with multiplicative interaction effects when relationship between independent variable and dependent variable are non-additive.
My question is, Are all models with multiplicative interaction effects non-linear? and all models with additive interaction effects linear?
Also, With non-linearity, the effect of independent variable on dependent variable depends on the value of independent variable, in effect, independent variable somehow interacts with itself. Does that mean that an independent variable(x1) interacts with itself(x1)? or does it mean that independent variables(x1) interact with other independent variables(x2, x3…) and not with itself?
I am confused with concepts of of linearity, non-linearity, additivity and non-additivity (multiplicativity).
Any help is much appreciated.
Best Answer
we use models with multiplicative interaction effects when relationship between independent variable and dependent variable are non-additive.
The answer to such a question depends on what you mean when you say 'linear' and 'nonlinear', and what domain of models you're restricting yourself to.
Usually the terms 'linear' and 'nonlinear' in statistical models refers to linearity in the parameters, not the variables.
So for example, $y = \alpha x^2 +\epsilon$ is linear in $\alpha$ though not in $x$, while $y = \exp(-\alpha) x +\epsilon$ is non-linear in $\alpha$, though it is in $x$. In usual parlance, the first is a linear model and the second is not. However, in those cases at least both may be turned into models that are linear in both the parameters and the predictors - in the first case by the transformation $x^* = x^2$, giving a model that has a linear relationship between $y$ and $x^*$, and in the second case by the reparameterization $\alpha^* = \exp(-\alpha)$.
As such a standard general linear model (regression-type model) with multiplicative interaction is linear in the parameters, even though it's not linear in either predictor (IV). However, note that even in terms of the IVs, it is conditionally linear - fix one of the IVs and the relationship is linear in the other.
[Minor mathematical aside: It should be noted that when we're taking about the relationships of $y$ and some $x$ being linear (in this sense rather than the 'makes a straight-line' sense), if we recognize we're using homogeneous co-ordinates in regression, it is linear. I mention it because I have seen people with enough mathematical background to be familiar with the mathematical definition of linearity object that 'linear regression is not linear'.]
If I understand what you're even asking with 'additive interaction effects', there's really no such thing. If it's additive it's already in the main effects and there's nothing left over for some notional 'interaction'.
Only if you think of 'effect' as inherently linear
This way lies much confusion. Why not just think of there being a relationship that's described by some curve rather than by a straight line?
--
Edit to address followup questions:
When you said "all models with multiplicative interaction effects" you presumably meant 'all models' in some class, such as regression models, or general linear models, or generalized linear models, or ... the list could go on for some time.
Me too.
In terms of $x$'s - the actual columns of the $X$-matrix in a regression - it's linear in that linked mathematical sense if you realize you're working with homogeneous co-ordinates.
A multiple linear regression is already linear in the mean parameters (i.e. the $\beta$ vector, the parameters other than $\sigma^2$), without any such need to invoke homogeneous co-ordinates. That I was referring to the relationship with the $x$'s when I raised homogeneous co-ordinates was explicitly stated.
Nope. The way you phrased the question I was responding to only makes sense if you take the word 'effect' to imply linearity, otherwise the whole notion of 'interaction with itself' seems to be utterly meaningless. How is one to interpret the phrase?
I regard the statement as an unhelpful attempt at analogy, and, as already explained, I think you should not think about it this way. Not everything that someone writes down is useful.
I'm not going to make any further attempt to interpret something that doesn't really make sense as a bare, general, statement without first having more clarification of its intent. I've suggested a way to interpret it that makes at least a little sense. If you want to interpret it more generally, explaining what it means would be up to you - or the original author of it.
I expect if you were to ask "what, exactly, does it mean*, you would receive an answer that contained a number of hidden premises, and one of those premises would rely, directly or indirectly, on taking underlying meaning of 'effects' to be linear, when we have no good reason to do that.