Solved – Set of uncorrelated but linearly dependent variables

correlationmathematical-statisticsmulticollinearity

Is it possible to have a set of $K$ variables that are uncorrelated but linearly dependent?

i.e.
$cor(x_i, x_j)=0$ and $ \sum_{i=1}^K a_ix_i=0$

If yes can you write an example ?

EDIT:
From the answers it follows that it is not possible.

Would it at least be possible that $\mathbb{P}(|\hat \rho_{x_i, x_j}-\hat \rho_{x_i, v}|<\epsilon)$ where $\hat\rho$ is the estimated correlation coefficient estimated from $n$ samples of the variables and $v$ is a variable that is uncorrelated with $x_i$.

I am thinking something like $x_K=\dfrac{1}{K} \sum_{i=1}^{K-1} x_i$ $K>>0$

Best Answer

As @RUser4512's answer shows, uncorrelated random variables cannot be linearly dependent. But, nearly uncorrelated random variables can be linearly dependent, and one example of these is something dear to the statistician's heart.

Suppose that $\{X_i\}_{i=1}^K$ is a set of $K$ uncorrelated unit-variance random variables with common mean $\mu$. Define $Y_i = X_i - \bar{X}$ where $\bar{X} = \frac 1K \sum_{i=1}^K X_i$. Then, the $Y_i$ are zero-mean random variables such that $\sum_{i=1}^K Y_i = 0$, that is, they are linearly dependent. Now, $$Y_i = \frac{K-1}{K} X_i - \frac 1K\sum_{j \neq i}X_j$$ so that $$\operatorname{var}(Y_i) = \left(\frac{K-1}{K}\right)^2+\frac{K-1}{K^2} = \frac{K-1}{K}$$ while $$\operatorname{cov}(Y_i,Y_j) = -2\left(\frac{K-1}{K}\right)\frac 1K + \frac{K-2}{K^2}= \frac{-1}{K}$$ showing that the $Y_i$ are nearly uncorrelated random variables with correlation coefficient $\displaystyle -\frac{1}{K-1}$.

See also this earlier answer of mine.

Related Question