I have 2 IID random variables $X_1$ and $X_2$ with $Bern(p)$ distribution. I have another binary random variable $Y$ taking values in $\{0,1\}$.
I am interested in comparing the following 2 mutual information $I(X_1+X_2;Y)$ and $I(2X_1;Y)$. Note that $Y=0$ with probability $\frac{1}{x+5}$ when the input ($X_1+X_2$ or $2X_1$) takes the value $x$.
I have a feeling that $I(X_1+X_2;Y) \leq I(2X_1;Y)$. Can someone help me prove or disprove this?
Best Answer
Your feeling is correct.
Indeed, the difference
$$d(p):=I(X_1+X_2;Y)-I(2X_1;Y)$$ between $I(X_1+X_2;Y)$ and $I(2X_1;Y)$ depends only on $p\in[0,1]$. The expression for $d(p)$ is somewhat complicated, containing a number of logarithmic terms; see the images below of a Mathematica notebook with details of calculations.
However, $d'''(p)$ is a rational function of $p$, which is rather easy to see to be $>0$ on $(0,1)$, so that $d''$ is increasing on $[0, 1]$. Also, $d''(0)>0$, so that $d''>0$ on $[0,1]$ and hence $d$ is strictly convex on $[0, 1]$. Finally, $d(0)=d(1)=0$ and hence $d<0$ on $(0,1)$.
That is, $I(X_1+X_2;Y)<I(2X_1;Y)$ for $p\in(0,1)$ and $I(X_1+X_2;Y)=I(2X_1;Y)$ for $p\in\{0,1\}$.