Solved – Estimating joint distribution from observed samples of the marginals

joint distributionnormal distributionrandom variable

Let's say I have two (normal) random variables $A1$ and $A2$ and during a random process, at each time instant, I am collecting a sample for each one of them.

Given the samples I collected, I can calculate two normal distributions to describe $A1$ and $A2$, respectively $\mathcal{N}(\mu_1, \sigma_1)$ and $\mathcal{N}(\mu_2, \sigma_2)$.

I want to know, given my estimated Gaussians and the samples I collected, how to calculate the joint probability distribution of the random vector $(A1, A2)$.

Best Answer

You can't compute joint distribution from marginals. Check this thread for much simper case with computing joint probability from individual probabilities.

In case of normal distributions, as in your question, imagine that you have two marginal distributions, each normal. Say that you are in lucky situation, that you know in advance that their joint distribution is bivariate normal, know the means and variances and the only thing that is unknown to you is the correlation parameter $\rho$. Unfortunately, if you know only marginals then you can't say anything about $\rho$, since there can be any correlation between the two variables, so there is unlimited number of bivariate normal distributions that can lead to such marginals. Moreover, it can be even more extreme and the variables can be marginally normal, but jointly not.