correlationindependence
Independence between random variables $X$ and $Y$ implies that $\text{Corr}\left(f(X),g(Y)\right)=0$ for arbitrary functions $f(\cdot)$ and $g(\cdot)$ (here is a related thread).
But is the following statement, or a similar one (perhaps more rigorously defined), correct?
If $\text{Corr}\left(f(X),g(Y)\right)=0$ for all possible functions $f(\cdot)$ and $g(\cdot)$, then $X$ and $Y$ are independent.
There's some merit to the idea, but it's not quantitatively correct.
The standard (Pearson) definition uses the expected product of the standardized values of $X$ and $Y$ to measure the correlation of $(X,Y)$. Specifically, recenter both $X$ and $Y$ (so that we're always discussing how much they differ from their respective means) and let their standard deviations be $\sigma$ and $\tau$, respectively. Then when $F$ is the bivariate distribution of $(X,Y)$,
$$\rho_F = \mathbb{E}_F\left(\frac{X}{\sigma}\frac{Y}{\tau}\right) = \mathbb{E}_F\left(r\left(\frac{X}{\sigma},\frac{Y}{\tau}\right)\right)$$
for the function $r(u,v)=uv.$ This function $r$ is positive when $u$ and $v$ have the same sign (are "aligned") and is negative otherwise. It grows in direct proportion to the values of $u$ and $v$. Thus its average value (the expectation) reflects a value-weighted balance of aligned and unaligned variations from the central point.
We could make $r$ less sensitive to the values of $u$ and $v$. A fairly extreme way to do this would be to give it a unit value when $u$ and $v$ are aligned and negate it when they are not; that is,
$$r^\prime(u,v) = \text{sgn}(u)\text{sgn}(v).$$
One could use this in place of $r$ in the correlation definition to create a robust measure of correlation of empirical distributions. In so doing, we would have the interpretation suggested in the question: the expectation of $r^\prime(X/\sigma,Y/\tau)$ is the chance that $X$ and $Y$ are "aligned" minus the chance they are not aligned. (In practice, the variables would be recentered using robust estimates such as their medians rather than their means and their standard deviations would be replaced by robust estimates of variation such as interquartile ranges or MADs.)
Yes, we could even choose $r^{\prime\prime}$ to be the indicator that $X$ and $Y$ are aligned, so that $\rho_F^{\prime\prime}$ really would be the chance of alignment. In so doing, however, we could only achieve values between $0$ and $1$: that can scarcely be interpreted like the Pearson coefficient, which ranges from $1$ down to $-1$. Since
$$r^{\prime\prime}(u,v) = \frac{1}{2} + \frac{r^\prime(u,v)}{2},$$
there is a clear, simple relationship between the characterization in the question (which appears to refer to $r^{\prime\prime}$) and the more familiar-looking correlation values attained by $r^\prime$. I will therefore continue to discuss $r^\prime$.
To appreciate the distinction between $\rho$ and $\rho^\prime$, consider the archetypical application of correlation: the bivariate normal distribution. When this particular distribution has Pearson correlation $\rho_F$, the alternative measure of correlation is
$$\rho^\prime_F = \frac{1}{2} + \frac{1}{\pi} \arctan\left(\frac{\rho_F}{\sqrt{1-\rho_F^2}}\right).$$
For values of $\rho$ within the range $(-0.6,0.6)$ or so, this function is approximately linear:
$$\rho^\prime_F \approx \frac{1}{2} + \frac{\rho_F}{\pi}.$$
For instance, with $\rho=0.6,$ $\rho^\prime = 0.705.$ This clearly differs (a lot) from the formula quoted in the question, which asserts $\rho^\prime_F \approx \rho_F$. This will only be the case for values of $\rho_F$ close to $\frac{\pi}{2(\pi-1)}\approx 0.733.$
The solid line graphs $\rho^\prime$ in terms of $\rho$ for bivariate Normal distributions. The dashed line is the linear approximation to $\rho^\prime$ around $\rho=0$. It has slope $\frac{1}{\pi}\approx 0.318$.
This one-to-one correspondence between $\rho$ and $\rho^\prime$ for the bivariate Normal distribution shows we could use either definition equally well for describing correlations where a Normal model applies--but I suspect the sampling distribution of $\rho^\prime$ might be a little more difficult to derive. There is not necessarily such a one-to-one correspondence between $\rho$ and $\rho^\prime$ among all bivariate distributions, though: for a given value of $\rho^\prime$ it is clear we could vary $\rho$ quite a bit by pulling a tiny bit of probability mass around in the $(X,Y)$ plane at large distances from the origin without changing $\rho^\prime$ at all.
Yes, because
$$\text{Corr}(X,Y)\ne0 \Rightarrow \text{Cov}(X,Y)\ne0$$
$$\Rightarrow E(XY) - E(X)E(Y) \ne 0 $$
$$\Rightarrow \int \int xyf_{X,Y}(x,y)dxdy -\int xf_X(x) dx\int yf_Y(y)dy \ne 0$$
$$\Rightarrow \int \int xyf_{X,Y}(x,y)dxdy -\int \int xyf_X(x) f_Y(y)dxdy \ne 0$$
$$\Rightarrow \int \int xy \big[f_{X,Y}(x,y) -f_X(x) f_Y(y)\big]dxdy \ne 0$$
which would be impossible if $f_{X,Y}(x,y) -f_X(x) f_Y(y) =0,\;\; \forall \{x,y\}$. So
$$\text{Corr}(X,Y)\ne0 \Rightarrow \exists \{x,y\}:f_{X,Y}(x,y) \ne f_X(x) f_Y(y)$$
Question: what happens with random variables that have no densities?
Best Answer
Using indicator functions of measurable sets like$$f(x)=\mathbb I_A(x)\quad g(x)=\mathbb I_B(x)$$leads to$$\text{cov}(f(X),g(Y))=\mathbb P(X\in A,Y\in B)-\mathbb P(X\in A)\mathbb P(Y\in B)$$therefore implying independence. As shown in the following snapshot of A. Dembo's probability course, proving the result for indicator functions is enough.
This is due to this monotone class theorem: