I have ordinal data from 3 related samples. First i conducted a Friedman test to know if there were differences between at least two samples. Next I conducted a Wilcoxon signed rank test to know between which of the 3 samples there were differences and i generated 3 possibles pairs.
However i don't know if i need to apply Bonferroni correction and basically compare my Asymp Sig (2 tailed) with P=0.05/3 instead of 0.05, or if i should mantain a p=0.05 to know if i accept or reject my null hypotesys.
Could anyone help me to clarify this question?
Thank you very much.
Best Answer
It depends on what kind of false discovery probability you wish to control.
Keeping p=0.05 means you allow 5% probability of false discovery (e.g. rejecting null hypothesis while it shouldn't be rejected) in each comparison.
Taking Bonferroni correction means that you allow 5% probability for at least one false discovery.
If the latter is what you deserve, I'd recommend using some other correction (eg. Holm), because Bonferroni is very conservative. It actally dumps false discovery probability far below 5%. But, on the other hand, with 3 comparisons only this effect is not very big.
Holm correction:
First, order your p-values in incereasing order. Let $p_1$ be the smallest and $p_n$ the largest one.
Then compare each $p_i$ with $$\frac{\alpha}{n-i+1}$$
So, in your case ($n=3$):
the smallest p-value ($p_1$) is significant if it's lower than 0.05/3,
the middle one ($p_2$) is significant if it's lower than 0.05/2,
and the largest ($p_3$) is significant if it's lower than 0.05.
Plus, some people say that if you find $p_i$ significant you should also consider all lower p-values significant. So according to them, if, for example, $p_3$ is significant, while $p_2$ is not, you should also treat $p_2$ as significant.