Hypothesis Testing – How to Test for a Monotonic Relationship Between Two Variables

correlationhypothesis testingspearman-rho

I'd like to test the hypothesis that there is a monotonic relationship between two variables, without assuming a specific model. What is the most robust (i.e. lowest probability of type-II error) way to do this?

I can think of a few options:

use a linear model of untransformed data. It'll be robust enough, even if I don't think the true relationship is linear.
look at rank-transformed data, e.g. with Spearman's rank correlation coefficient
use some kind of resampling approach, in which the order of the dependent variable is randomly shuffled. I'm not sure what statistic to compare in this approach.

Is there a fairly standard approach to this problem?

Best Answer

Spearman's or Kendall's correlations are the standard way to do this.

Related Solutions

Solved – How to test for a difference between ordered groups

Spearman correlation is fine as far as it goes, but don't stop there. What if there is a nonlinear relationship? E.g., perhaps the cost difference between those who rate the toy bad vs. medium is not comparable to the cost difference between those who rate it medium vs. good. An ANOVA would help you detect this. There's a reason to use ANOVA instead of 2 T-tests. Others might explain it better, but in a nutshell, the combination of omnibus test and (if significant) post hoc tests preserves Type I and Type II error rates better than the 2 T-tests would.

R – Identifying Issues with Spearman Correlation in Presence of Many Ties

Use a permutation test. You only need to permute one of the variables independently of the other; here, the response is permuted. Because the relationship in the example is strong, only a small number of permutations are needed (1000 in the example below).

As always, the actual statistic is compared to the distribution of permuted statistics. The p-value is the estimate of the tail probability of the permutation distribution relative to the actual statistic. In some cases the test statistic has a discrete distribution, so it's wise to check the frequencies with which (a) the permutation statistics strictly exceed the actual statistic and (b) the permutation statistics equal or exceed the actual statistic. The code illustrates this by splitting the difference.

test <- function(y) suppressWarnings(cor.test(x, y, method="spearman")$estimate)
rho <- test(y)                                     # Test statistic
p <- replicate(10^3, test(sample(y, length(y))))   # Simulated permutation distribution

p.out <- sum(abs(p) > rho)    # Count of strict (absolute) exceedances
p.at <- sum(abs(p) == rho)    # Count of equalities, if any
(p.out + p.at /2) / length(p) # Proportion of exceedances: the p-value.

suppressWarnings quiets any complaints from cor.test that it cannot compute a p-value due to ties.

Best Answer

Related Solutions

Solved – How to test for a difference between ordered groups

R – Identifying Issues with Spearman Correlation in Presence of Many Ties

Related Question