Friedman Test – Performing a Post Hoc Test for Friedman Test

friedman testpost-hoctukey-hsd-test

I am running a Friedman test for my non-Gaussian data, on MATLAB. However, I am confused about the post hoc test to choose. Two concerns:

(1) MATLAB's multcompare function's default would be Tukey HSD, but I read in multiple sites saying that it's only for parametric test (?). Link to the MATLAB guide on multcompare. If this is the case then it's quite silly that MATLAB suggested to do multiple comparison following Friedman. Other options would be Bonferroni method, Dunn and Sidák’s approach, Fisher's least significant difference procedure or Scheffé's S procedure

(2) Would it be somehow better to do a Wilcoxon test and correct to the number of hypothesis (eg in my case, 6) using Bonferonni's method?

Please if anyone could enlighten me with this

Best Answer

So:

  • Fisher LSD, Tukey HSD and Scheffe test are all parametric, so this is not an option for you
  • p-value corrections like Bonferroni, Dunn-Sidak and many others (Holm, Benjamini-Hochberg, ...) always can be used
  • beware of Bonferroni, it is the most conservative correction (it can dump your type-I error far below 5%)

If you are looking for post-hoc test after Friedman test, Conover and Nemenyi procedures may be options to consider.

Here you can read about them.

Two notes:

  • this is a tutorial for R pacjage called PMCMR not for MATLAB (but contains some formulas to give you some insight into those tests)
  • if you switch to R, you sholud use PMCMRplus package as PMCMR is outdated and no longer maintained