we use models with multiplicative interaction effects when relationship between independent variable and dependent variable are non-additive.
My question is, Are all models with multiplicative interaction effects non-linear? and all models with additive interaction effects linear?
The answer to such a question depends on what you mean when you say 'linear' and 'nonlinear', and what domain of models you're restricting yourself to.
Usually the terms 'linear' and 'nonlinear' in statistical models refers to linearity in the parameters, not the variables.
So for example, $y = \alpha x^2 +\epsilon$ is linear in $\alpha$ though not in $x$, while $y = \exp(-\alpha) x +\epsilon$ is non-linear in $\alpha$, though it is in $x$. In usual parlance, the first is a linear model and the second is not. However, in those cases at least both may be turned into models that are linear in both the parameters and the predictors - in the first case by the transformation $x^* = x^2$, giving a model that has a linear relationship between $y$ and $x^*$, and in the second case by the reparameterization $\alpha^* = \exp(-\alpha)$.
As such a standard general linear model (regression-type model) with multiplicative interaction is linear in the parameters, even though it's not linear in either predictor (IV). However, note that even in terms of the IVs, it is conditionally linear - fix one of the IVs and the relationship is linear in the other.
[Minor mathematical aside: It should be noted that when we're taking about the relationships of $y$ and some $x$ being linear (in this sense rather than the 'makes a straight-line' sense), if we recognize we're using homogeneous co-ordinates in regression, it is linear. I mention it because I have seen people with enough mathematical background to be familiar with the mathematical definition of linearity object that 'linear regression is not linear'.]
all models with additive interaction effects linear?
If I understand what you're even asking with 'additive interaction effects', there's really no such thing. If it's additive it's already in the main effects and there's nothing left over for some notional 'interaction'.
Also, With non-linearity, the effect of independent variable on dependent variable depends on the value of independent variable,
Only if you think of 'effect' as inherently linear
in effect, independent variable somehow interacts with itself.
This way lies much confusion. Why not just think of there being a relationship that's described by some curve rather than by a straight line?
--
Edit to address followup questions:
What do you mean when you say "what domain of models you're restricting yourself to"?
When you said "all models with multiplicative interaction effects" you presumably meant 'all models' in some class, such as regression models, or general linear models, or generalized linear models, or ... the list could go on for some time.
Thanks! for noting about linearity. For the longest time, even I thought being linear meant the relationship was a straight line.
Me too.
This does clear some doubts, but raises a few questions. So, if we recognize we're using homogeneous co-ordinates in regression, it is linear.
In terms of $x$'s - the actual columns of the $X$-matrix in a regression - it's linear in that linked mathematical sense if you realize you're working with homogeneous co-ordinates.
A multiple linear regression is already linear in the mean parameters (i.e. the $\beta$ vector, the parameters other than $\sigma^2$), without any such need to invoke homogeneous co-ordinates. That I was referring to the relationship with the $x$'s when I raised homogeneous co-ordinates was explicitly stated.
Also, Did you mean to say "Only if you think of 'effect' as inherently nonlinear instead of linear?
Nope. The way you phrased the question I was responding to only makes sense if you take the word 'effect' to imply linearity, otherwise the whole notion of 'interaction with itself' seems to be utterly meaningless. How is one to interpret the phrase?
What I meant to ask was. I read somewhere that "with non-linearity, the effect of X on Y depends on the value of X and X somehow interacts with itself".
I regard the statement as an unhelpful attempt at analogy, and, as already explained, I think you should not think about it this way. Not everything that someone writes down is useful.
Does this mean that X interacts with itself(X)? or does it mean that X interact with other variables(X,W etc.) if any?
I'm not going to make any further attempt to interpret something that doesn't really make sense as a bare, general, statement without first having more clarification of its intent. I've suggested a way to interpret it that makes at least a little sense. If you want to interpret it more generally, explaining what it means would be up to you - or the original author of it.
I expect if you were to ask "what, exactly, does it mean*, you would receive an answer that contained a number of hidden premises, and one of those premises would rely, directly or indirectly, on taking underlying meaning of 'effects' to be linear, when we have no good reason to do that.
Your argument is correct. Since the true relationship is linear, the square and cubic terms in the cubic fit are not significant. And that is confirmed by the large p-values (0.215 and 0.461).
As mentioned by @mark999, it seems you are using a very large sample size because the standard errors are so small. If you follow the original question and use $n=100$, the numbers would be different but the conclusion will stay the same.
> set.seed(1)
> x <- runif(100, 0, 5)
> y <- 5 + 2*x + rnorm(x)
> linear.fit <- lm(y ~ x)
> cubic.fit <- lm(y ~ x + I(x^2) + I(x^3))
> summary(linear.fit)
Call:
lm(formula = y ~ x)
Residuals:
Min 1Q Median 3Q Max
-1.84978 -0.56222 -0.08707 0.52427 2.51661
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.82067 0.20581 23.42 <2e-16 ***
x 2.06247 0.07069 29.18 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.9411 on 98 degrees of freedom
Multiple R-squared: 0.8968, Adjusted R-squared: 0.8957
F-statistic: 851.2 on 1 and 98 DF, p-value: < 2.2e-16
> summary(cubic.fit)
Call:
lm(formula = y ~ x + I(x^2) + I(x^3))
Residuals:
Min 1Q Median 3Q Max
-1.85466 -0.59246 -0.09722 0.54144 2.48360
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.63484 0.46455 9.977 < 2e-16 ***
x 2.42391 0.74492 3.254 0.00157 **
I(x^2) -0.16671 0.34102 -0.489 0.62607
I(x^3) 0.02144 0.04535 0.473 0.63742
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.9496 on 96 degrees of freedom
Multiple R-squared: 0.897, Adjusted R-squared: 0.8938
F-statistic: 278.7 on 3 and 96 DF, p-value: < 2.2e-16
Best Answer
You can use Linear Regression to model any linear/non-linear relationship using basis expansion (slides from Elements of Statistical Learning). In your case you could probably exponentiate some of the variables, but it might be preferable to use an automatic method, such as Multivariate Adaptive Regression Splines, that still provides interpretable results.