Solved – Spearman Correlation Coefficient is more than one

correlationspearman-rho

I am comparing the ranking of two lists. The lists are not very nicely correlated and the number of items is very large (6630). Is it possible to have a correlation coefficient of more than one. I followed the following link:

https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient#Definition_and_calculation

My code gives correct result for the hours and tv example. However, when I run the same on the large list, the ranking correlation is more than one. I am not able to figure out why.

Best Answer

Note that from the formula on the page you link to the sample value of $\rho_s$ is $1-\frac{\sum_i 6d_i^2}{n(n^2-1)}$.

Consequently, to get a value bigger than 1, $\sum d_i^2$ would have to be negative (which is not possible).

This suggests a problem with the calculation.

Related Solutions

Solved – Spearman’s rank correlation for a beginner: Sample size and Interpretation

If you are concerned about sample size and significance, good concepts to start out with include effect size and power (while on the topic of CIs, you might want to include accuracy as well)

As noted previously, 95% CI refers not to probability but to confidence; it is not the likelihood that the current CI contains the population parameter, but that out of 100 CIs 95 CIs will succeed in capturing the population parameter. The true probability remains unknown (unless you take a Bayesian approach), see Explorations in statistics: confidence intervals.

Spearman Correlation – Dealing with Problems in Spearman Correlation Due to Many Ties

While ranking the data for use in Spearman correlation is possible with Excel formulas (like almost everything), it is not that easy.

I would suggest a little easier solution, that at the moment will work only in 32-bit Excel: use RExcel:

First you'd need to download and install the R 2.15.2 for Windows.
Then Open the R prompt and copy & paste the following code (which will install, among others, components that will allow quite seamless communication with Excel. All software has uninstallers in case you decide to remove them later):

makeSureInstalled<-function(package)
{
   if(length(grep(paste("^",package,"$",sep=""),noquote(installed.packages())[,1]))==0)
      install.packages(package)
library(package=package,character.only=TRUE)
}
makeSureInstalled("rcom")
installstatconnDCOM()
comRegisterServer()
comRegisterRegistry()
makeSureInstalled("RExcelInstaller")
installRExcel(ForegroundServer=TRUE)

Then open your Excel, paste your data (I'll assume it comes in two collumns)
On new ribbon you should click "Start R":
Put this formulas:
On cell H8 you will have the p-value.
If you want to have the Spearman correlation coefficient $\rho$, type in this formula:

=REval("cor.test(var1,var2,method='spearman')$estimate")

Best Answer

Related Solutions

Solved – Spearman’s rank correlation for a beginner: Sample size and Interpretation

Spearman Correlation – Dealing with Problems in Spearman Correlation Due to Many Ties

Related Question