Spearman rho vs Kendall tau. These two are so much computationally different that you cannot directly compare their magnitudes. Spearman is usually higher by 1/4 to 1/3 and this makes one incorrectly conclude that Spearman is "better" for a particular dataset. The difference between rho and tau is in their ideology, proportion-of-variance for rho and probability for tau. Rho is a usual Pearson r applied for ranked data, and like r, is more sensitive to points with large moments (that is, deviations from cloud centre) than to points with small moments. Therefore rho is quite sensitive to the shape of the cloud after the ranking done: the coefficient for an oblong rhombic cloud will be higher than the coefficient for an oblong dumbbelled cloud (because sharp edges of the first are large moments). Tau is an extension of Gamma and is equally sensitive to all the data points, so it is less sensitive to peculiarities in shape of the ranked cloud. Tau is more "general" than rho, for rho is warranted only when you believe the underlying (model, or functional in population) relationship between the variables is strictly monotonic. While Tau allows for nonmonotonic underlying curve and measures which monotonic "trend", positive or negative, prevails there overall. Rho is comparable with r in magnitude; tau is not.
Kendall tau as Gamma. Tau is just a standardized form of Gamma. Several related measures all have numerator $P-Q$ but differ in normalizing denominator:
- Gamma: $P+Q$
- Somers' D("x dependent"): $P+Q+T_x$
- Somers' D("y dependent"): $P+Q+T_y$
- Somers' D("symmetric"): arithmetic mean of the above two
- Kendall's Tau-b corr. (most suitable for square tables): geometric mean of those two
- Kendall's Tau-c corr. (most suitable for rectangular tables): $N^2(k-1)/(2k)$
- Kendall's Tau-a corr. (makes nо adjustment for ties): $N(N-1)/2 = P+Q+T_x+T_y+T_{xy}$
where $P$ - number of pairs of observations with "concordance", $Q$ - with "inversion"; $T_x$ - number of ties by variable X, $T_y$ - by variable Y, $T_{xy}$ – by both variables; $N$ - number of observations, $k$ - number of distinct values in that variable where this number is less.
Thus, tau is directly comparable in theory and magnitude with Gamma. Rho is directly comparable in theory and magnitude with Pearson $r$. Nick Stauner's nice answer here tells how it is possible to compare rho and tau indirectly.
See also about tau and rho.
Note that since the denominator only depends on the margins, not on the association, the exact permutation p-value is the same for any of the statistics that has $N_c-N_d$ on the numerator (Kendall's tau, tau-b, tau-c, Somer's-D etc).
There are algorithms for computing p-values of all these measures of association for ordinal data efficiently, by taking account of the ordering of the permutations induced by some simple form of the test statistic, and only considering the ones more extreme than the test statistic (or considering as few additional ones as possible), generally arising from ideas based on Mehta and Patel's networking algorithm, though there have been developments and ideas from many authors.
Some stats packages implement such algorithms. It might be worth checking whether some of the exact-test related packages already have an efficient version of it for some version of the Kendall statistic.
For example, I believe SPSS Exact Tests has this implemented for the Kendall tau.
(Out of curiosity, why would it matter for you if you were only able to get a probabilistic bound on the p-value using resampling? How does (say) knowing the p-value is almost certainly less that $3.2 \times 10^{-5}$ rather than computing it to be exactly $1.6245 \times 10^{-6}$? What additional information does that give you?)
Best Answer
I found that Spearman correlation is mostly used in place of usual linear correlation when working with integer valued scores on a measurement scale, when it has a moderate number of possible scores or when we don't want to make rely on assumptions about the bivariate relationships. As compared to Pearson coefficient, the interpretation of Kendall's tau seems to me less direct than that of Spearman's rho, in the sense that it quantifies the difference between the % of concordant and discordant pairs among all possible pairwise events. In my understanding, Kendall's tau more closely resembles Goodman-Kruskal Gamma.
I just browsed an article from Larry Winner in the J. Statistics Educ. (2006) which discusses the use of both measures, NASCAR Winston Cup Race Results for 1975-2003.
I also found @onestop answer about Pearson's or Spearman's correlation with non-normal data interesting in this respect.
Of note, Kendall's tau (the a version) has connection to Somers' D (and Harrell's C) used for predictive modelling (see e.g., Interpretation of Somers’ D under four simple models by RB Newson and reference 6 therein, and articles by Newson published in the Stata Journal 2006). An overview of rank-sum tests is provided in Efficient Calculation of Jackknife Confidence Intervals for Rank Statistics, that was published in the JSS (2006).