I am trying to understand whether I can calculate the joint probability distribution for three random variables when I know two joint probability distribution of two random variables.
Define a probability space: $\left( \Omega, ,\mathcal{F}, \mathcal{P} \right)$ and three random variables $\mathbf{s}: \Omega \rightarrow S$, $\mathbf{x_1}: \Omega \rightarrow X$ $\mathbf{x_2}: \Omega \rightarrow X$. Suppose the two joint probability distributions $\mathcal{P}_{\mathbf{x_1}, \mathbf{s}}$ and $\mathcal{P}_{\mathbf{x_2}, \mathbf{s}}$ are known. As an example, suppose $\left\lbrace s_1, s_2 \right\rbrace = S$ and $\left\lbrace x_1, x_2 \right\rbrace = X$. The probability distribution for $\mathcal{P}_{\mathbf{x_1}, \mathbf{s}}$ is:
\begin{array}{ l | l| l| }
& s_1 & s_2 \\ \hline
x_1 & 2/3 & 1/3 \\ \hline
x_2 & 1/3 & 2/3 \\
\hline
\end{array}
The probability distribution for $\mathcal{P}_{\mathbf{x_2}, \mathbf{s}}$ is:
\begin{array}{ l | l| l| }
& s_1 & s_2 \\ \hline
x_1 & 1/2 & 1/2 \\ \hline
x_2 & 1/2 & 1/2 \\
\hline
\end{array}
Given this information, can I compute the probability distribution $\mathcal{P}_{\mathbf{x_1}, \mathbf{x_2}, \mathbf{s}}$? What other information would I need to compute the joint distribution?
Best Answer
You can't. Here are two possible joint distributions which both yield the correct marginal distributions from your question, but which are different.
$$\begin{array}{l|l|l|} & \mathbf{s}=s_1 & \mathbf{s}=s_2 \\ \hline (\mathbf{u},\mathbf{v})=(x_1,x_1) & \frac{2}{6} & \frac{1}{6} \\ (\mathbf{u},\mathbf{v})=(x_1,x_2) & \frac{2}{6} & \frac{1}{6} \\ (\mathbf{u},\mathbf{v})=(x_2,x_1) & \frac{1}{6} & \frac{2}{6} \\ (\mathbf{u},\mathbf{v})=(x_2,x_2) & \frac{1}{6} & \frac{2}{6} \\ \end{array}$$
$$\begin{array}{l|l|l|} & \mathbf{s}=s_1 & \mathbf{s}=s_2 \\ \hline (\mathbf{u},\mathbf{v})=(x_1,x_1) & \frac{1}{6} & \frac{2}{6} \\ (\mathbf{u},\mathbf{v})=(x_1,x_2) & \frac{3}{6} & 0 \\ (\mathbf{u},\mathbf{v})=(x_2,x_1) & \frac{2}{6} & \frac{1}{6} \\ (\mathbf{u},\mathbf{v})=(x_2,x_2) & 0 & \frac{3}{6} \\ \end{array}$$
(Note that I renamed your random variable $\mathbf{x}_1$ to $\mathbf{u}$ and $\mathbf{x}_2$ to $\mathbf{v}$. Using the same name for random variable and for the elements of their image, i.e. $X = (x_1,x_2)$ seemed confusing)