Solved – How to calculate the effect size for the Kolmogorov-Smirnov Z statistic

effect-sizekolmogorov-smirnov test

is there a way of calculating an effect size for the Kolmogorov-Smirnov Z statistic (in SPSS or by hand)? Or should I stick to the Mann-Whitney test, even though my group sizes are less than n=25?

Best Answer

Yes. $D = Z/\sqrt{n}$ for the one-sample test. $D = Z/\sqrt{\frac{n_1 n_2}{n_1 + n_2}}$ for the two-sample test. $D$ should also be the "Most Extreme Differences - Absolute" entry in the output graphic (double-click the table shown in the SPSS output viewer). $Z$ might be labeled "Test Statistic," "Kolmogorov-Smirnov Z," or something else depending on which test and version of SPSS you're using.
It depends. Mann-Whitney tests for a difference in the central tendencies by comparing average ranks; K-S tests for a difference in distributions by comparing the maximum difference in empirical cumulative distribution functions. If you expect strong shape differences, such as only low and high values in one group but middle values for the other group (this would be atypical for most data), K-S is a better choice. If you expect just a location shift, Mann-Whitney is more powerful.

Related Solutions

Solved – Two sample Kolmogorov-Smirnov test and p-value interpretation

If you are using the traditional 0.05 alpha level cutoff then all but group 3 are significantly different from your full group. It is a little easier to see this if the p-values are not in scientific notation ( you can use options(scipen=5) in R to make this less likely). Also group 1 becomes non-significant for some adjustments for multiple tests. You should consider whether that adjustment applies in your case or not. Also note that the groups that are not significant could be different, just low power.

But that just means that any differences, however small, are not easily explained by chance. It could be that your groups are close enough for practical purposes. It is usualy more meaningful to plot the data to see how different the distributions are. You could use the qqplot function as one approach. The vis.test function in the TeachingDemos package for R gives another approach.

One possible hitch is if your groups are part of the "Full" data set as well, then you don't have the independence assumed (but given the sample sizes, I am not sure how much this would affect things). You could address this by taking random samples from the full data set and computing the KS-distance for each (ignore the p-value), then compare where your actual data falls relative to the random samples.

Most of this comes down to what question you really want answered, many of the exact distributional tests answer a different question than the researcher is really interested in.

Kolmogorov-Smirnov Test – How to Perform a Kolmogorov-Smirnov Two-Sample Test

I am assuming you are asking because the Suanshu help page reports in reference to the K-S distribution, "This is not done yet." Luckily, it is very easy to do in R. If x and y are your two samples, ks.test(x,y) returns the test statistic and pvalue. For example,

> x <- rnorm(50)
> y <- runif(30)
> ks.test(x, y)    
        Two-sample Kolmogorov-Smirnov test    
data:  x and y 
D = 0.5, p-value = 9.065e-05
alternative hypothesis: two-sided

By default, it will compute exact or asymptotic p-values based on the product of the sample sizes (exact p-values for n.x*n.y < 10000 in the two-sample case), or you can specify this option with a third argument, exact=F or exact=T. Exact p-values are calculated using the methods of Marsaglia, et al. (2003), which the Suanshu documentation also cites. Some large sample approximations are given here, although I don't have a proper citation. Lastly, if you don't want to install R, there are web calculators for the two-sample K-S test, although I don't know if they use the same algorithm as R because the one I found only reported three decimal points for the p-value.

Best Answer

Related Solutions

Solved – Two sample Kolmogorov-Smirnov test and p-value interpretation

Kolmogorov-Smirnov Test – How to Perform a Kolmogorov-Smirnov Two-Sample Test

Related Question