I am trying to understand whether it is possible and if so then how to find a joint distribution given marginals and a correlation matrix.
In particular, suppose $X_{1},\ldots,X_{n}$ are discrete random variables. I am given their marginals $p_{X_{1}},\ldots,p_{X_{n}}$ and their correlation matrix. Can I construct (uniquely?) the joint $p_{X_{1},\ldots,X_{n}}$?
I tried this for $n=2$, assuming each of the two random variables has two possible outcomes $a$ and $b>a$, and assuming that the marginals of $X_{1}$ and $X_{2}$ are the same given by $\mathbb{P}[X_{1}=a]=c$. Let $r$ be the correlation of $X_{1}$ and $X_{2}$. My Mathematica readily calculates unique joint distribution provided $c(1-r)<1$ and claims there is no joint otherwise.
How would I generalize to more random variables with more than two possible outcomes? What I do in Mathematica (calculate correlation from joint distribution and then solve for the joint distribution that gives $r$) does not generalize nicely. Also, are there some results that would tell me what I am doing can/cannot be done? Uniquely?
I tried reading StackExchange questions/answers here, here, here, here, and here. But none of the answers help me forward. Also, I know that this question is trivial when the random variables are independent. So let's disregard that special case.
Best Answer
In general, the joint distribution is not uniquely determined by the marginals. The correlations only give second order moment information so are still not enough to pin down a unique joint distribution.
Adapting this example from wiki, suppose $W$ is a random variable that takes values $-1,1$ with equal probability, and $X\sim N(0,1)$. Assume $X$ is independent of $W$. Define $Y=WX$. You can show $X,Y$ are uncorrelated and are marginally standard normal but not independent. However, two iid standard normals are also uncorrelated.
Here is a discrete example with infinitely many joint distributions that yield the same uncorrelated marginal distributions:
Suppose $X\in \{-1,0,1\}$ and $Y\in \{0,1\}$. For some $p\in (1/6,1/3)$, consider the joint distribution $$\begin{align} P(X=-1,Y=1)&=P(X=1,Y=1)=p,\\ P(X=-1,Y=0)&=P(X=1,Y=0)=1/3-p,\\ P(X=0,Y=0)&=2p-1/3\\ P(X=0,Y=1)&=2/3-2p\\ \end{align} $$
The marginals are given by
$$P(X=x)=1/3 \quad \forall x\in \{-1,0,1\}\\ P(Y=0)=1/3,P(Y=1)=2/3$$
and $X,Y$ are uncorrelated (note $X$ has mean zero):
$$E[XY]=(-1)(1)p+(1)(1)p=0.$$