Solved – the distinction between “relationships” and “differences” (particularly in the context of group differences on a numeric attribute)

association-measurecorrelationgroup-differencest-testterminology

What is the distinction between a relationship and a difference in the context of statistics – if you have a nominal variable and an interval variable, when should you use a correlation to measure the relationship, rather than a t-test or an ANOVA?

I'm not after the usual 'what is your question about – relationships or differences' (although I obviously appreciate the importance), but rather what is it about the data or the constructs that I should know to make this decision.

If for example you have males and females and their intelligence scores. Is there a statistical/conceptual reason why a t-test is better here than a point-biserial correlation?

This is probably going to be an obvious question. To be honest though, I don't feel like I've ever really understood the distinction, so perhaps some of you can articulate it for me?

Best Answer

Association (or "correlation") and difference are two sides of the same coin. Difference between groups by some quantitative characteristic can be reasoned as the association between variables "group" and "characteristic". Eta coefficient, also called correlation ratio, is the proper association measure between a nominal variable and a scale variable, and is therefore the other side of the coin for ANOVA or t-test. In fact, Eta is used as the effect-size measure (i.e. standardized measure of the size of the difference) in ANOVA. Association measures which vary from 0 to 1 or -1 to 1 are often used as effect size measures. Point-biserial correlation is equal to Eta when there are 2 groups. Point-biserial correlation is just Pearson r.

Related Solutions

Solved – How to test hypothesis of no group differences

I think you are asking about testing for equivalence. Essentially you need to decide how large a difference is acceptable for you to still conclude that the two groups are effectively equivalent. That decision defines the 95% (or other) confidence interval limits, and sample size calculations are made on this basis.

There is a whole book on the topic.

A very common clinical "equivalent" of equivalence tests is a non-inferiority test/ trial. In this case you "prefer" one group over the other (an established treatment) and design your test to show that the new treatment is not inferior to the established treatment at some level of statistical evidence.

I think I need to credit Harvey Motulsky for the GraphPad.com site (under "Library").

Solved – Significant difference between difference scores of 2 groups on two measures

To me, your hypothesis sounds like an interaction effect of the two factors "version" (of questionnaire) and "sex" (of participant). As the data are dependent / repeated measures, this should be taken into account by means of a multilevel aka hierachical aka mixed model. Instead of a linear model, you should use something like a logit link (cf. logistic regression) for your binary DV. Suppose your data are in long format, i. e., two rows for every participant, the first for the old version, the second for the new version. The DV is experience of crime 0/1. Data for 3 participants could like this:

     id sex version DV
[1,]  1   1       0  1
[2,]  1   1       1  0
[3,]  2   0       0  1
[4,]  2   0       1  1
[5,]  3   0       0  1
[6,]  3   0       1  0

According to your hypothesis, you predict a cross-level interaction between the level 1-predictor "version" and the level 2-predictor "sex" (in ANOVA / repeated measures terms, this is an interaction between a within- and a between-factor). If you code your variables like this sex: female=1, male=0; version: old=1, new=0, then you probably predict a positive beta of the interaction and you could test this one-sided. Code in R using the package lme4 could look like this (could be done with any multilevel software):

m1 <- lmer(DV ~ version*sex + (1|id), data=mydata, family=binomial(link="logit"))
summary(m1)

HTH

Best Answer

Related Solutions

Solved – How to test hypothesis of no group differences

Solved – Significant difference between difference scores of 2 groups on two measures

Related Question