Solved – Interpreting significance of predictor vs significance of predictor coeffs in multinomial logistic regression

logisticstatistical significance

I have a multinomial logistic regression with dependent variable valued in {-1,0,1} (reference category is 0) and a number of continuous and discrete predictors. After running the regression a continuous predictor of interest ('size') has a Type 3 analysis of effects p-value of 0.0683, and the two coefficients (corresponding to outcomes of -1 and 1) have p-values of 0.8786 and 0.0220 respectively.

I read somewhere that one should only look at the significance of the coefficients if the predictor itself is significant at the chosen level. Is this right? My naive sense is that the predictor is borderline (taking alpha=0.05 for argument's sake), and that 'size' has a significant relationship to outcome=1 but not to outcome = -1. I would say that the significance of the relationship to outcome=1 is not terribly strong, but that is ok for the application in mind (or at least, with the indirect data I am forced to use)

Best Answer

The p-value itself cannot tell you how strong the relationship is, because the p-value is so influenced by sample size, among other things. But assuming your N is something on the order of 100-150, I'd say there's a reasonably strong effect involving Size whereby as Size increases, the log of the odds of Y being 1 is notably different from the log of the odds of Y being 0. As you indicate, the same cannot be said of the comparison of Y values of -1 and 0.

You are right in viewing all of this as somewhat invalidated by the overall nonsignificance of Size (depending on your alpha, or criterion for significance). You wouldn't get too many arguments if you simply declared Size a nonfactor due to its high p-value. But then again, if your N is sufficiently small--perhaps below 80 or 100--then your design affords low power for detecting effects, and you might make a case for taking seriously the specific effect that managed to show up anyway.

A way around the problem of relying on p-values involves two steps. First, decide what range of odds ratios would constitute an effect worth bothering with, or worth calling substantial. (The trick there is in being facile enough with odds to recognize what they mean for the more intuitive metric of probability.) Then construct a confidence interval for the odds ratio associated with each coefficient and consider it in light of your hypothetical range. Regardless of statistical significance, does the effect have practical significance?

Related Solutions

Solved – Interpreting the changing significance of a variable in a Logistic Regression

I would interpret your results as follows: The first model shows that the likelihood of getting the outcome (dependent variable = 1) decreases as the years of parenting increase. The odds ratio is simply the exponentiated regression coefficient ($OR=\exp(-0.532)=0.587$) and if years of parenting are continuous, represents the odds ratio per 1 unit (I assume per 1 year) of parenting. It doesn't matter whether you look at the regression coefficient or at the odds ratio to see how the relationship is: if the regression coefficient is negative, the odds ratio is < 1 and this denotes a negative relationship and vice versa. The coefficient of similarity index is positive and this means that the likelihood of getting the outcome is increasing with increasing similarity index (odds ratio = 1.08). This site offers a good starting point on how to interpret logistic regression results. This site here even discusses it with an annotated SPSS output.

In the second model containing the interaction, the interaction is significant whereas the main effect is not any more. This has been discussed in this post, for example. Interactions mean that the slope (on the log-odds scale) for years parenting differ for different levels of similarity index. Just because the inclusion of the interaction term made your main effect insignificant, you should not exclude the main effect. See here on how to report interactions in logistic regressions (this paper seems very useful too). You should also ask the question whether an interaction makes sense from a scientific point of view.

Solved – Interpretation of logistic regression intercept with one dumthe coded categorical variable

I think you are making this hard on yourself. Make sure race is a factor variable so that the software provides the overall $\chi^2$ of association with $k-1$ d.f. for $k$ categories. Coding doesn't affect the value of $\chi^2$. Don't use a stepwise process for making inference about the importance of race. Use the overall "chunk" test as described above, which has a built-in perfect multiplicity adjustment besides being invariant to coding. In R this would look like (for a binary or ordinal logistic model predicting $Y$):

require(rms)
f <- lrm(Y ~ rcs(age, 4) + race)
anova(f)   # 3 d.f. test for age, k-1 for race
# also prints 2 d.f. test of linearity in age
# age fit is restricted cubic spline with 4 default knots

When doing multiple imputation with the Hmisc package aregImpute function or with the mice package, you would substitute the following for the 2nd line above:

f <- fit.mult.impute(Y ~ rcs(age, 4) + race, lrm, impute_object, n.impute=20)

which would adjust the covariance matrix for multiple imputation [n.impute recommended to be the percent of observations that have any variable missing].

Best Answer

Related Solutions

Solved – Interpreting the changing significance of a variable in a Logistic Regression

Solved – Interpretation of logistic regression intercept with one dumthe coded categorical variable

Related Question