Solved – Perfectly correlated (normal) random variables

correlationnormal distributionprobability

I am not sure in the terminology, so I will simply try to explain the situation that I would like to model as I see it. Suppose there is a set of random variables. The variables are correlated in such a way that they deviate from their expected values into the same direction all together. By this I mean that they can be either all together larger then their expectation or all together lower. Is it possible to model such a dependency with a multivariate normal random variable $\mathbf{X} \sim \mathcal{N}(\mathbf{\mu}, \mathbf{\Sigma})$, assuming the knowledge about the marginal distributions of the components $\mathbf{X}_i \sim \mathcal{N}(\mu_i, \sigma^2_i)$? How to construct $\mathbf{\Sigma}$ is this situation? Thank you.

Best wishes,
Ivan

Best Answer

Consider this very simple snippet:

    m1 <- 0
    m2 <- 0
    cov <- 0.8
    x1 <- rnorm(100, mean=m1)
    x2 <- cov*x1 + rnorm(100,mean=m2-cov*m1,sd=sqrt(1-cov*cov))
    plot(x1,x2)
    x2a <- x2*sign(x1-m1)*sign(x2-m2)
    plot(x1,x2a)

It folds the distribution of x2 around its mean, aligning its deviations from the mean to those of x1 from its mean. Of course the resulting distribution cannot be characterized as a multivariate normal, although each margin is normal:

    plot( density(x1), ylim=c(0,0.5) )
    hist( x1, add=T, prob=T )

Contour

Contours of the density of (x1, x2a): the probability that would ordinarily be associated with values in quadrants II or IV has been symmetrically displaced into quadrants I and III, leaving the marginal distributions undisturbed.

This is a classic (counter)example of a distribution that has normal margins, yet is not a multivariate normal; frankly, I don't know how to build any other ones.

The transformation increases the correlation somewhat:

    > cor(x1,x2)
    [1] 0.7999774
    > cor(x1,x2a)
    [1] 0.8575814

You would've seen a much stronger effect with lower cov, of course: you can start with cov=0 and still get the correlation of the resulting variables above 0.6.

Related Solutions

Normal Distribution – Linear Combination of Dependent Multivariate Variables

In that case, you have to write (with hopefully clear notations) $$ \left(\begin{matrix}X\\Y \end{matrix}\right) \sim \mathcal{N}\left[ \left(\begin{matrix}\mu_X\\\mu_Y\end{matrix}\right), \Sigma_{X,Y} \right] $$ (edited: assuming joint normality of $(X,Y)$) Then $$ AX+BY=\left(\begin{matrix}A& B \end{matrix}\right) \left(\begin{matrix}X\\Y \end{matrix}\right) $$ and $$ AX+BY+C \sim \mathcal{N}\left[ \left(\begin{matrix}A& B \end{matrix}\right) \left(\begin{matrix}\mu_X\\\mu_Y\end{matrix}\right) + C, \left(\begin{matrix}A & B \end{matrix}\right)\Sigma_{X,Y} \left(\begin{matrix}A^T \\ B^T \end{matrix}\right)\right] $$ i.e. $$ AX+BY+C \sim \mathcal{N}\left[A\mu_X + B\mu_Y +C, A\Sigma_{XX}A^T+B\Sigma_{XY}^TA^T+A\Sigma_{XY}B^T+B\Sigma_{YY}B^T \right] $$

Solved – Given a multivariate normal distribution, how can we simulate uniform random variables that hold on to the correlation structure

Wikipedia introduces the method of drawing values from the distribution. I am now trying to illustrate it in more details.

If we want to draw sample from the multivariate normal distribution specified by $\mu \in \mathbb{R}^{n\times 1}$ and $\Sigma \in \mathbb{R}^{n\times n}$ and $\Sigma^\top = \Sigma$. We can do the following to generate random variable $\mathbf{x} \in \mathbb{R}^{n\times 1}$ such that $\mathbf{x} \sim \mathcal{N}(\mu, \Sigma)$.

Find matrix $A$ such that $A A^\top = \Sigma$. When $\Sigma$ is positive definite, the Cholesky decomposition is often used. Other way of getting such matrix $A$ includes SVD decomposition.
When we have such matrix $A$, we just use some random number generator, e.g. randn(n, 1) function in Matlab to generate $n$ independent standard normal variables and get a random vector $\mathbf{z} \in \mathbb{R}^{n \times 1}$, and $\mathbf{x} = \mu + A \mathbf{z}$ is what we want.

Julia code for this is

n = 2                  # 2 dimension situation
μ = [1.; 2.]           # mean value
Σ = [exp(0) exp(-1);
    exp(-1) exp(0)]
Σ = Symmetric(Σ)
A = chol(Σ)'           # chol(Σ) = A^T
z = randn(2, 2000)     # generate 2000 points
x = A * z .+ μ

We now verify that data generated in this way satisfies that $$\mathbf{x} \sim \mathcal{N}(\mu, \Sigma)$$ As $\mathbf{z} \sim \mathcal{N}(0, 1)$, from affine transformation of multivariate normal distribution we know that $\mathbf{x} = \mu + A \mathbf{z}$ indeed satisfies the needed property.

Best Answer

Related Solutions

Normal Distribution – Linear Combination of Dependent Multivariate Variables

Solved – Given a multivariate normal distribution, how can we simulate uniform random variables that hold on to the correlation structure

Related Question