I have several datasets of independent variables that have a monotonic (but non-linear) relationship. If I want to assess if they're correlated, the test of choice is Spearman's (rho) or Kendall's (tau) rank correlation coefficients.
Yet, sometimes I've observed a slight U-shape distribution in scatter plots, in what I suspect to be non-monotonic datasets.
I have a number of questions:
- Is there a way to test if my data is monotonic prior to Spearman's rho / Kendall's tau correlation calculations?
- Is it possible to decompose my dataset into monotonic sections, to analyse them separately?
- Is there any equivalent to Spearman's rho test (or Kendall's tau) that accounts for multiple monotonic components?
I'm not sure if the last question makes sense.
Thanks a lot.
Best Answer
You could plot the data and look for a non-monotone shape.
Also, you could fit a generalized additive model (GAM) which estimates nonparametric functions of the predictor variables. This can be done in the
mgcv
package in R. For example:which produces:
Note that
So, both Spearman's rho and Kendall's tau are not helpful.
Now, if we run a GAM, we get
With edf>1 there is evidence of non-linearity in the data, which doesn't prove that the association is non-monotonic, but nevertheless suggests that it might be.
Yes ! Sticking with the same dataset, we can do:
which gives:
and this handles the first segment of the data, then:
which gives:
So here we can see a strong negative association in the first segment and a strong positive association in the second.
Not that I am aware of.