Effect Size – Calculating Pairwise Effect Size for Dunn’s Test in R

effect-sizer

I am using ggsttaplot – I am curious how to get effect size for each pair:

# install.packages("tidyverse")  # for everything ;)
library(tidyverse)

# install.packages("ISLR")
library(ISLR)

# install.packages("ggstatsplot")
library(ggstatsplot)

# stabilize the output of "sample_n()"
set.seed(1)
d <- Wage %>% group_by(education) %>% sample_n(50, replace = TRUE)

p<- ggbetweenstats(
  data = d,
  x    = education,
  y    = wage,
  type = "nonparametric")
p

# a list of tibbles containing statistical analysis summaries
extract_stats(p)[1]

The plot shows epsilon suare:0.3 but I am interested in effect size for each pair. I cant find any function or library that will calculate it

Best Answer

I assume you're using Dunn (1964) test, that would be used as a post-hoc for a Kruskal-Wallis test ?

One approach would be to use an effect size statistic that's appropriate for a Wilcoxon-Mann-Whitney test, in a pairwise manner. These effect size statistics include Vargha and Delaney’s A, Cliff’s delta, and Glass rank biserial correlation coefficient, among others.

With the caveat that I wrote it, there is a function in the rcompanion package that does just this.

Y = c(1,2,3,2,3,4,4,5,6)
Group = c(rep("A",3), rep("B", 3), rep("C", 3))
Data = data.frame(Group, Y)

library(rcompanion)

multiVDA(Y ~ Group, data=Data)

  ###   Comparison    VDA     CD     rg  VDA.m  CD.m  rg.m
  ### 1      A - B 0.2220 -0.556 -0.556 0.7780 0.556 0.556
  ### 2      A - C 0.0000 -1.000 -1.000 1.0000 1.000 1.000
  ### 3      B - C 0.0556 -0.889 -0.889 0.9444 0.889 0.889

Addition:

A few useful references for relevant effect size statistics.

Tomczak and Tomczak. 2014. The need to report effect size estimates revisited. Trends in Sport Sciences 1(21). www.tss.awf.poznan.pl/files/3_Trends_Vol21_2014__no1_20.pdf

King, B.M., P.J. Rosopa, and E.W. Minium. 2000. Statistical Reasoning in the Behavioral Sciences, 6th. Wiley.

Grissom, R.J. and J.J. Kim. 2011. Effect Sizes for Research: Univariate and Multivariate Applications, 2nd. Routledge.

Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences, 2nd Edition. Routledge.

Vargha, A. and H.D. Delaney. A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong. 2000. Journal of Educational and Behavioral Statistics 25(2):101–132.

My own thoughts:

Mangiafico, S. 2016. "Two-sample Mann–Whitney U Test" in Summary and Analysis of Extension Program Evaluation in R. rcompanion.org/handbook/F_04.html.

Mangiafico, S. 2016. "Kruskal–Wallis Test" in Summary and Analysis of Extension Program Evaluation in R. rcompanion.org/handbook/F_08.html

Related Question