Solved – Interpretation of Kolmogorov-Smirnov test result

kolmogorov-smirnov test

How do I interpret the following output of the Kolmogorov-Smirnov [KS] test? [shown below, with reproducible example in R]

Also: Can the KS test be used for predictive model diagnostics?

x <- c(7.6,8.4,8.6,8.7,9.3,9.9,10.1,10.6,11.2) 
y <- c(5.2,5.7,5.9,6.5,6.8,8.2,9.1,9.8,10.8,11.3,11.5,12.3,12.5,13.4,14.6)
ks.test(x,y)

Resulting output:

Two-sample Kolmogorov-Smirnov test

data: x and y D = 0.4, p-value = 0.2653 alternative hypothesis:
two-sided

Best Answer

A P-Value of greater than 0.05 means that you cannot reject the null hypthesis that the two underlying distributions do not defer. <0.05 means that you can reject the null hypothesis that the distributions do not differ

Related Solutions

Solved – Two sample Kolmogorov-Smirnov test and p-value interpretation

If you are using the traditional 0.05 alpha level cutoff then all but group 3 are significantly different from your full group. It is a little easier to see this if the p-values are not in scientific notation ( you can use options(scipen=5) in R to make this less likely). Also group 1 becomes non-significant for some adjustments for multiple tests. You should consider whether that adjustment applies in your case or not. Also note that the groups that are not significant could be different, just low power.

But that just means that any differences, however small, are not easily explained by chance. It could be that your groups are close enough for practical purposes. It is usualy more meaningful to plot the data to see how different the distributions are. You could use the qqplot function as one approach. The vis.test function in the TeachingDemos package for R gives another approach.

One possible hitch is if your groups are part of the "Full" data set as well, then you don't have the independence assumed (but given the sample sizes, I am not sure how much this would affect things). You could address this by taking random samples from the full data set and computing the KS-distance for each (ignore the p-value), then compare where your actual data falls relative to the random samples.

Most of this comes down to what question you really want answered, many of the exact distributional tests answer a different question than the researcher is really interested in.

Kolmogorov-Smirnov Test – How to Perform a Kolmogorov-Smirnov Two-Sample Test

I am assuming you are asking because the Suanshu help page reports in reference to the K-S distribution, "This is not done yet." Luckily, it is very easy to do in R. If x and y are your two samples, ks.test(x,y) returns the test statistic and pvalue. For example,

> x <- rnorm(50)
> y <- runif(30)
> ks.test(x, y)    
        Two-sample Kolmogorov-Smirnov test    
data:  x and y 
D = 0.5, p-value = 9.065e-05
alternative hypothesis: two-sided

By default, it will compute exact or asymptotic p-values based on the product of the sample sizes (exact p-values for n.x*n.y < 10000 in the two-sample case), or you can specify this option with a third argument, exact=F or exact=T. Exact p-values are calculated using the methods of Marsaglia, et al. (2003), which the Suanshu documentation also cites. Some large sample approximations are given here, although I don't have a proper citation. Lastly, if you don't want to install R, there are web calculators for the two-sample K-S test, although I don't know if they use the same algorithm as R because the one I found only reported three decimal points for the p-value.

Best Answer

Related Solutions

Solved – Two sample Kolmogorov-Smirnov test and p-value interpretation

Kolmogorov-Smirnov Test – How to Perform a Kolmogorov-Smirnov Two-Sample Test

Related Question