Say that you have four random $A$, $B$, $C$, and $D$. I would like to know if the inequality
$$
I(A,B;C,D)\geq I(A;C)+I(B;D)
$$
holds.
Since I have not found the inequality in literature, I assume it does not hold. But I would like an example or some intuition as to why it does not. It must have to do with information being redundant, but I cannot see how.
If $A,C$ is independent of $B,D$ then the inequality holds trivially with equality. And the weaker statements
$$
I(A,B;C,D)\geq I(A;C)
$$
and
$$
I(A,B;C,D)\geq I(B;D)
$$
follow from the data processing inequality.
Thanks in advance for any help.
Best Answer
It doesn't hold. Counterexample:
Using $H(A)+H(B)-H(A,B)=I(A;B)$ and $H(A|C)=I(A;D|C)+H(A|C,D)$, let
$$\begin{align} d &= I(A,B;C,D) - I(A;C)-I(B;D)\\ &=H(A,B)- H(A,B|C,D) - H(A) + H(A|C)-H(B)+H(B|D)\\ &= -I(A;B) +I(A;D|C)+H(A|C,D)+ I(B;C|D)+H(B|C,D)-H(A,B|C,D)\\ &= -I(A;B) + I(A;D|C)+ I(B;C|D) + I(A;B|C,D) \end{align} $$
Now, suppose $A\to C \to D \to B$ form a Markov chain.
Then, in general, $I(A;B) >0$ but $I(A;D|C)=I(B;C|D)=I(A;B|C,D)=0$
Then $d<0$ and the conjecture is false.