Correlation – How to Generate Correlated Random Variables

correlationcovariancenormal distribution

If we have 2 normal, uncorrelated random variables $X_1, X_2$ then we can create 2 correlated random variables with the formula

$Y=\rho X_1+ \sqrt{1-\rho^2} X_2$

and then $Y$ will have a correlation $\rho$ with $X_1$.

Can someone explain where this formula comes from?

Best Answer

Suppose you want to find a linear combination of $X_1$ and $X_2$ such that

$$ \text{corr}(\alpha X_1 + \beta X_2, X_1) = \rho $$

Notice that if you multiply both $\alpha$ and $\beta$ by the same (non-zero) constant, the correlation will not change. Thus, we're going to add a condition to preserve variance: $\text{var}(\alpha X_1 + \beta X_2) = \text{var}(X_1)$

This is equivalent to

$$ \rho = \frac{\text{cov}(\alpha X_1 + \beta X_2, X_1)}{\sqrt{\text{var}(\alpha X_1 + \beta X_2) \text{var}(X_1)}} = \frac{\alpha \overbrace{\text{cov}(X_1, X_1)}^{=\text{var}(X_1)} + \overbrace{\beta \text{cov}(X_2, X_1)}^{=0}}{\sqrt{\text{var}(\alpha X_1 + \beta X_2) \text{var}(X_1)}} = \alpha \sqrt{\frac{\text{var}(X_1)}{\alpha^2 \text{var}(X_1) + \beta^2 \text{var}(X_2)}} $$

Assuming both random variables have the same variance (this is a crucial assumption!) ($\text{var}(X_1) = \text{var}(X_2)$), we get

$$ \rho \sqrt{\alpha^2 + \beta^2} = \alpha $$

There are many solutions to this equation, so it's time to recall variance-preserving condition:

$$ \text{var}(X_1) = \text{var}(\alpha X_1 + \beta X_2) = \alpha^2 \text{var}(X_1) + \beta^2 \text{var}(X_2) \Rightarrow \alpha^2 + \beta^2 = 1 $$

And this leads us to

$$ \alpha = \rho \\ \beta = \pm \sqrt{1-\rho^2} $$

UPD. Regarding the second question: yes, this is known as whitening.

Related Question