Solved – Friedman test and post-hoc test for Python

friedman testgeneralized-estimating-equationsnonparametricpost-hocpython

In my dataset, I have five (ordinal) groups with an x-amount of measurement. Because homoscedasticity is violated, I performed the Friedman chi-square test to see if there are any statistical differences between the groups:

fried = stats.friedmanchisquare(*[grp for idx, grp in df.iteritems()]))

This returned a statistical difference, but now I would like to find out between which groups the differences exist. In R there is a nice solution for this (Friedman's test and post-hoc analysis, https://www.r-statistics.com/2010/02/post-hoc-analysis-for-friedmans-test-r-code/), where they use the Wilcoxon-Nemenyi-McDonald-Thompson test, but I am unable to find one for Python.

Is there a possibility to do post-hoc analyses for the Friedman test? Alternatively, what would we be a good alternative for the Friedman test that does allow me to compare between groups, e.g. a generalized estimating equation?

Best Answer

I am currently looking into this issue myself; according to this paper there are a number of possibilities to perform posthoc-tests (Update: an extension regarding the use of non-parametric tests can be found here):

  • Perform the Nemenyi-test for all pairwise combinations; this is similar to the Tukey-test for ANOVA.
  • Perform the Bonferroni-Dunn-test; in this setting one compares all values to a list of control values.
  • Alternatively, one can perform step-up and step-down procedures sequentially testing hypotheses ordered by their significance. One can use Holm's step-down procedure, Hochberg's step-up procedure or Hommel's procedure.

The STAC Python library seems to include all these tests, except for Hommel's procedure.

Related Question