[Math] Joint distribution of dependent Bernoulli Random variables

probabilityprobability distributions

I have $N$ Bernoulli random variables $X_1, …, X_{N}$ with known parameters $p_1, …, p_{N}$. I want generate a joint distribution in which these random variables are not independent as I know that joint distribution would just be the product of their marginals.

How can I create this joint distribution that can be updated as I add more Bernoulli random variables?

Dependent Bernoulli trials

^ This seems to be my exact question but I am looking for a much simpler way of defining the distribution than selecting $2^n – 1$ parameters

Best Answer

Based on your most recent comment, I think you should consider a 2-state Markov chain to produce a sequence of random variables $X_i,$ taking values in $\{0, 1\},$ roughly as follows:

Start with a deterministic or random $X_1.$ Then

(i) $P\{X_{i+1} = 1|X_i = 0\} = \alpha,$ and (ii) $P\{X_{i+1} = 0|X_i = 1\} = \beta.$

The parameters $\alpha$ and $\beta$ are the respective probabilities of 'changing state' from one $X_i$ to the next. To avoid certain kinds of deterministic sequences, you may want to use $0 < \alpha, \beta < 1.$ If $\alpha = 1 - \beta,$ then the sequence is independent.

By induction, one can show that $$P\{X_{1+r} = 0|X_1 = 0\} = \frac{\beta}{\alpha+\beta} + \frac{\alpha(1-\alpha - \beta)^r}{\alpha+\beta}.$$ If $|1-\alpha - \beta| < 1$, then in the long run $P\{X_n = 0\} \approx \beta/(\alpha+\beta),$ regardless of the value of $X_1.$

Moreover, there are similar formulas for the '$r$-step transitions' from 0 to 1, 1 to 0, and 1 to 1. Of course, I am skipping over a lot of detail here.

Perhaps there is a rich enough variety of models here to satisfy your curiosity as to what happens when independence fails in this way.

Later chapters in many probability books have a complete development of the theory of 2-state Markov chains. Also there are several good elemeentary books just on Markov chains. [Google '2-state Markov Chain'. One reference among many is Chapter 6 of Suess and Trumbo (2010), Springer, in which I have a personal interest.]

Related Question