You would use the Bonferroni for a one-way test. But let's be clear:
You would not use the Bonferroni adjustment on the Kruskal-Wallis test itself. The Kruskal-Wallis test is an omnibus test, controlling for an overall false-positive rate.
You would use the Bonferroni for post hoc Dunn's pairwise tests. Indeed, Dunn introduced the "Bonferroni" adjustment.
You could also consider using more powerful family-wise error rate adjustment methods, such as the Holm-Sidak method, or still more-powerful false discovery rate adjustment methods, such as the Benjamini-Yekutieli adjustment. These and other multiple comparisons adjustments are implemented specifically for Dunn's test in Stata (within Stata type net describe dunntest, from(https://alexisdinno.com/stata)
), and in R (see http://cran.r-project.org/web/packages/dunn.test/).
You might also consider using the more powerful Conover-Iman post hoc (only) test statistic. This test is implemented for Stata (within Stata type net describe conovertest, from(https://alexisdinno.com/stata)
), and for R in the conover.test package, and includes the same selection of multiple comparisons adjustment procedures.
EDIT: I missed that you seem to be wanting something like a multivariate (multiple dependent variables) nonparametric one-way ANOVA. Katz and McSweeney offer such a generalization of the Kruskal-Wallis test, although I am not aware of an implementation in software. They also provide post hoc Scheffé-like univariate and multivariate procedures, which ought to be amenable to multiple comparisons adjustments. Their test is likewise omnibus for all variables.
References
Conover, W. J. (1999). Practical Nonparametric Statistics. Wiley, Hoboken, NJ, 3rd edition.
Conover, W. J. and Iman, R. L. (1979). On multiple-comparisons procedures. Technical Report. LA-7677-MS, Los Alamos Scientific Laboratory.
Dunn, O. J. (1964). Multiple comparisons using rank sums. Technometrics, 6(3):241–252.
Katz, B. M. and Mcsweeney, M. (1980). A multivariate Kruskal-Wallis test with post hoc procedures. Multivariate Behavioral Research, 15:281–297.
Best Answer
Let's step back and look at what the data would look like. From what you describe, 3 algorithms (i.e. groups or treatments) and 10 datasets (i.e. subjects). In this case, you have a a within-subjects design (i.e. repeated measures) with one factor. One way to represent this is like this:
You will typically see examples that have 'subject' as a label. In your case, your 'subjects' are 'datasets'. If you can assume normality, you would do repeated-measures ANOVA. However, you state you know the accuracies are not normally distributed and you naturally want a non-parametric method. Your dataset is also balanced (10 samples/group) so we can use the Friedman test (which essentially is a nonparametric repeated-measures ANOVA).
If you get a significant p-value from the test, you would do post-hoc analysis with a pairwise paired Wilcoxon test with some sort of correction (e.g. bonferroni, holm, etc.). You would not use Mann-Whitney because you have 'paired/repeated measures' data.
Lastly, you probably want the effect size any significant differences. This also would use the wilcoxon test. In R there is no function I can recall right now but the equation is very simple:
$$r=\frac{Z}{sqrt(N)}$$
Where Z is the Z-score and N is the sample size (between the two groups being compared). You can get this Z-score using the
wilcoxsign_test
from thecoin
package.Using the above data, this can be done in R with the following. Please note, the above data was just randomly generated so there is no significance. This is just for demonstrating some code: