Several books define what conditional independence means, and some of them go on to use the term "conditionally i.i.d. random variables", but i could not find a precise definition of what it means for random variables to be "conditionally identically distributed".
If $X_1$, $X_2$ are conditionally identically distributed given the $\sigma$-algebra $\mathcal{A}$, does it mean
-
There is a regular version of $P\left(X_1\in\cdot\mid\mathcal{A}\right)$, $\kappa_1\left(B,\omega\right)$, and a regular version of $P\left(X_2\in\cdot\mid\mathcal{A}\right)$, $\kappa_2\left(B,\omega\right)$, such that for all $\omega$, $\kappa_1\left(\cdot,\omega\right)$ and $\kappa_2\left(\cdot,\omega\right)$ are the same probability measure? or
-
If $\kappa_1\left(B,\omega\right)$ is any version of $P\left(X_1\in\cdot\mid\mathcal{A}\right)$ (regular or not) and $\kappa_2\left(B,\omega\right)$ is any version of $P\left(X_2\in\cdot\mid\mathcal{A}\right)$ (likewise), then for all $B$, $k_1\left(B,\cdot\right)=k_2\left(B,\cdot\right)$ a.s.?
Best Answer
You are handed a coin. With probability $1/2$ it's a fair coin. With probability $1/2$ it's a biased coin that gives you $90\%$ heads and $10\%$ tails. Given that you get "heads" the first six times, what's the conditional probability that you get "heads" the seventh time? It's pretty high, because the conditional probability that you got the biased coin, given that outcome, is high. In other words, obviously the outcomes of the tosses are not independent.
But the outcomes of the tosses are
There's a conditional probability distribution of the sequence of outcomes, given that you got the fair coin. There's also a conditional probability distribution of the sequence of outcomes, given that you got the biased coin. In either of those distributions, you've got a sequence of i.i.d. outcomes.
So the outcomes are conditionally i.i.d. given the kind of coin you got.
More generally, suppose a coin gives you $100\cdot R\%$ heads, where $R$ is uniformly distributed between $0$ and $1$. The tosses are conditionally independent given $R$, but given that in the first trillion tosses you get $43\%$ heads, the probability that you get heads on the next toss is close to $0.43$, so the tosses are not independent. Nor even close to independent, as you see if you consider the conditional probability that the first outcome is heads, given that the second outcome is heads (which is $2/3$) versus the marginal probability that the first outcome is heads (which is $1/2$).