Just in case that someone (else) has to perform a comparison of correlation coefficients on multiple pairs of variables, here's a function based on rg255's helpful reply to copy:
cor.diff.test = function(x1, x2, y1, y2, method="pearson") {
cor1 = cor.test(x1, x2, method=method)
cor2 = cor.test(y1, y2, method=method)
r1 = cor1$estimate
r2 = cor2$estimate
n1 = sum(complete.cases(x1, x2))
n2 = sum(complete.cases(y1, y2))
fisher = ((0.5*log((1+r1)/(1-r1)))-(0.5*log((1+r2)/(1-r2))))/((1/(n1-3))+(1/(n2-3)))^0.5
p.value = (2*(1-pnorm(abs(fisher))))
result= list(
"cor1" = list(
"estimate" = as.numeric(cor1$estimate),
"p.value" = cor1$p.value,
"n" = n1
),
"cor2" = list(
"estimate" = as.numeric(cor2$estimate),
"p.value" = cor2$p.value,
"n" = n2
),
"p.value.twosided" = as.numeric(p.value),
"p.value.onesided" = as.numeric(p.value) / 2
)
cat(paste(sep="",
"cor1: r=", format(result$cor1$estimate, digits=3), ", p=", format(result$cor1$p.value, digits=3), ", n=", result$cor1$n, "\n",
"cor2: r=", format(result$cor2$estimate, digits=3), ", p=", format(result$cor2$p.value, digits=3), ", n=", result$cor2$n, "\n",
"diffence: p(one-sided)=", format(result$p.value.onesided, digits=3), ", p(two-sided)=", format(result$p.value.twosided, digits=3), "\n"
))
return(result);
}
(I assume you're talking about r's obtained from a sample.)
The test on that website applies in the sense that it treats r like any parameter whose value may differ between two populations. How is r any different from any other measure, such as the mean, which you're very confident in comparing, using the t-test? Well, it's different in that it's bound between -1,1, it doesn't have the proper distribution, so you need to Fisher transform it before doing inference (and back transform it afterwards, if you want to e.g. get a CI). The z scores resulting from the test do have the proper form to do inference on. That's what the test you're linking to is doing.
So what you link to is a procedure of inferring what might happen if you could get the r for the entirety of the population(s) from which you're sampling - would the r for one group be higher than for the other, or would they be precisely the same? Let's call this later hypothesis H$_0$. If the test returns a low p value, it implies that based on your sample, you should have little confidence in the hypothesis that the true value for the difference between the two r's would be exactly 0 (as such data would occur rarely if the difference in r was exactly 0). If not, you do not have the data to reject, with confidence, this hypothesis of precisely equal r, either because it is true and/or because your sample is insufficient.
Note that I could have done the same story about the difference in means (using the t-test), or any other measure.
A completely different question is if the difference between the two would be meaningful. Sadly, there is no straight-forward answer to this, and no statistical test can give you the answer. Maybe the true value (the population value, not the one you observe) of r is .5 in one, and .47 in the other group. In this case, the statistical hypothesis of their equivalence (our H$_0$) would be false. But is this a meaningful difference? It depends - is something on the order of 3% more explained variance meaningful, or meaningless?
Cohen has given rough guidelines for interpreting r (and presumably, differences between r's), but did so only under the advice that these are nothing but a starting point. And you do not even know the exact difference, even if you do some inference, e.g. by calculating the CI for the differences between the two correlations. Most likely, a range of possible differences will be compatible with your data.
A comparatively safe bet would be computing the confidence intervals for your r's and possibly the CI for their difference, and letting the reader decide.
Best Answer
There are various tests you can apply. Biedenhofen & Musch (2015, PLoS ONE) give pointers and describe the
cocorpackage for R, which implements these tests. You can also submit your correlations for testing to a web tool which internally uses thecocorpackage.