Suppose I know that, for two random variables $X,Y$, we have
$$Cov(X,Y)\neq 0.$$
What happens if we take a monotone transformation of $X$; will the inequality persist? That is, say $f(.)$ is a strictly increasing (not necessarily linear) function, then does it follow that
$$Cov(f(X),Y)\neq 0?$$
Since covariance depicts linear relationships, this would only follow if $f$ were linear, right? In that case, under which assumptions on the relation of $X,Y$ can we say that it does follow for nonlinear transformations $f$? (If it helps, I am currently looking at the case where $f$ is the cdf of the standard normal $\Phi$.)
Probability – Covariance of Two Random Variables with Monotone Transformation
probabilityprobability theorystatistics
Related Solutions
For jointly (per @Did) normal random variables, uncorrelated implies independent. In particular, it is easy to see that the joint density function factors, giving the product of the two marginal density functions.
Also, for normal data, the sample mean $\bar X$ and sample SD $S$ are independent. (Proof via linear algebra or moment generating functions.) But $\bar X$ and $S$ are not independent except for normal data.
In the left panel below $S$ is plotted against $\bar X$ for 30,000 randomly generated standard normal datasets of size $n = 5.$ As a 'naturally occurring' instance where zero correlation and dependence coexist: in the right panel the same is done for 30,000 samples of size $n = 5$ from $Beta(.5, .5).$ For these beta data $\bar X$ and $S$ are uncorrelated, but not independent.
m = 30000; n = 5
x = rnorm(m*n); NRM = matrix(x, nrow=m)
ax = rowMeans(NRM); sx = apply(NRM, 1, sd)
cor(ax, sx)
## -0.001177232 # consistent with uncorrelated
y = rbeta(m*n, .5, .5); BTA = matrix(y, nrow=m)
ay = rowMeans(BTA); sy = apply(BTA, 1, sd)
cor(ay, sy)
## -0.001677063 # consistent with uncorrelated
The function $f(x)=x(1-x)$ applied to $\min(x,y)$ is not a covariance function since it is not positive definite on $[0.5,1]$. To see this, just calculate the determinant of the "covariance" matrix $C$ of the points $v=0.6$ and $w=0.8$. You find $C(v,v)=0.6*0.4=0.24$, $C(w,w)=0.8*0.2=0.16$ and $C(v,w)=f(0.6)=0.24.$ Now $\det C= 0.24 * 0.16 - (0.24)^2<0.$
This example gives already a good idea of what is going on or better going wrong with non-monotonic $f$. In fact I claim:
If $f$ is such that there exist points $v,w$ with $v< w$ and $f(v)> f(w)$ then $C(x,y)=f(\min(x,y))$ is not positive definite.
The first necessary condition is $$C(x,x)=f(x)>0$$ for all $x$ in the domain of $f$. We assume this holds and calculate $$C(v,v)C(w,w) - C(v,w)^2=f(v)f(w) - f(v)^2 = f(v)( f(w) - f(v))<0$$ by assumption on $f$.
Best Answer
(I have edited the question changing $f$ to $g$ in order to keep the symbol $f$ for probability density functions). First, indeed, when the transformation is linear(affine actually), the covariance involving the transformation will continue to be non-zero. For non-linear transformations you cannot obtain very specific conditions. My approach is the following:
For compactness, set $Z=g\left(X\right)$. Then you require the conditions under which $$\operatorname{Cov}(Z,Y)\neq 0? \;\Rightarrow\; E\left(ZY\right)\,-\,E\left(Z\right)E\left(Y\right)\, \neq0?\qquad [1]$$
Using Integrals to express the expected values, we require (assuming certain regularity conditions) $$\int_{D_Z} \int_{D_Y} zyf_{Z,Y}\left(z,y\right)\,dy\,dz \;-\;\left(\int_{D_Z}zf_{Z}\left(z\right)dz\right) \left(\int_{D_Y} yf_{Y}\left(y\right)\,dy\right)\neq0?$$
$$\Rightarrow \int_{D_Z} \int_{D_Y} zyf_{Z,Y}\left(z,y\right)\,dy\,dz \;-\;\int_{D_Z} \int_{D_Y} zyf_{Z}\left(z\right)f_{Y}\left(y\right)\,dy\,dz\neq0?$$
$$\Rightarrow \int_{D_Z} \int_{D_Y} zy\left[f_{Z,Y}\left(z,y\right) \;-\;f_{Z}\left(z\right)f_{Y}\left(y\right)\right]\,dy\,dz\neq0?\qquad[2]$$
where the $f()$ functions are joint or marginal probability density functions, and $D$ denotes the support. Now define $\omega \equiv(z,y)$ and note that $zy\;$,$\,z\;$,$\;y\;$, can all be derived as linear transformations of $\omega$, say $zy=\tau_0(\omega)\;$,$\,z=\tau_1(\omega)\;$,$\;y=\tau_2(\omega)\;$. For example $z=\omega[1\; 0]'$, the prime denoting the transpose. Also define $C\equiv D_Z\times D_Y$. Then we can compact even further by writing $$\cdots\Rightarrow \int_C h\left[\omega,\tau_0(\omega),\tau_1(\omega),\tau_2(\omega)\right]d\omega\neq0?\qquad[3]$$
Why all this trouble? Because now it is clear that what you are asking is the requirements under which a double definite integral of a function of its variable does not equal zero when integrated over the whole domain of its variable. Reversely, we want to exclude the conditions under which this integral equals zero. First since $\operatorname{Cov}(X,Y)\neq 0\;$, it implies that the random variables $X$ and $Y$ are not independent. Therefore we have $f_{Z,Y}\left(z,y\right) \;\neq\;f_X \left(x\right)f_Y \left(y\right)$ . But this does not guarantee that the double integral will be non-zero. It may still equal zero if the integrand in eq.[2] is an odd function w.r.t. to at least one of $z$ or $y$, and if the domain w.r.t to this variable is symmetric around the origin (which will hold if for example $D= \Bbb R$ - extended). Assume that indeed $C=\Bbb R\times\Bbb R$ extended. Therefore we must guarantee that the integrand $zy\left[f_{Z,Y}\left(z,y\right) \;-\;f_{Z}\left(z\right)f_{Y}\left(y\right)\right]$ is not an odd function w.r.t. to either $z$ or $y$ separately. Now if you work the integrand, substituting $-z$ for $z$ etc, you will be able to reduce the condition into the following: "The non-linear transformation should not be such as to make both $f_{Z,Y}\left(z,y\right) \;$ and $f_{Z}\left(z\right)$ even functions of $z$." For $Y$ this could be stated as "If the pdf of $Y$ is an even function, then the non-linear transformation should not be such as to make $f_{Z,Y}\left(z,y\right) \;$ an even function of $y$.". Obviously this is distribution-specific (or at least distribution family- specific) and you cannot go more specific than that. If it looks disappointingly general, imagine a true jungle. Then independence is a straight clear path through this jungle, and linear dependence is an elliptical clear path approaching asymptotically the straight path of independence. The rest of the jungle is non-linear dependence. $$ \text{}$$ Afterthought: A less "mathematical" and more "statistical" condition (although not necessarily more useful) would be "The non-linear transformation must be such as to not render the variables $Z$ and $Y$ sub-independent". In a nutshell, sub-independent random variables are dependent but uncorrelated: their dependence is "perfectly non-linear".