I'm pretty new to probability and a question is giving me some troubles.
A binary information source produces $0$ and $1$ with equal probability. The output of the source, denoted as $X$, is transmitted via an additive white Gaussian noise (AWGN) channel. The output of the channel, denoted as $Y$, satisfies $Y = X + N$, where the random noise $N$ has the distribution $N(0,0.1)$.
a) If the channel outputs $0.2$, what is the probability that $0$ is the source output and what is the probability that $1$ is the source output
b) If the channel output is $0.2$ and you are required to make a guess on the source output, what is your guess and why?
I've tried solving this directly and looking at it generally, and from both perspectives it isn't making a lot of sense to me.
Directly, it's asking to find $P(X=0 \mid Y = 0.2)$ which I assume can be solved for like any other conditional problem, using
$$f_{X\mid Y}(x\mid y) = \frac{f(x,y)}{f_Y(y)}$$
Since X seems like it's best expressed using a Bernoulli distribution, I don't know how to actually evaluate this at all. To find the joint PDF of X and Y (which I think I need to find the conditional probability of X given Y = y) I would need to combine a Bernoulli distribution and a Normal distribution. I can't find any examples of this and I'm not entirely sure it's possible at all seeing as one is discrete and one is continuous. This is where my solution falls apart.
Looking at it more generally though, I have a feeling that there might not be a solution at all. The source is perfectly binary, so it will only produce EXACTLY 0 or 1. And the output from the channel has to be EXACTLY 0.2. For that to be true, doesn't the noise have to be EXACTLY 0.2 or 0.8? And since the noise is a continuous distribution, isn't the probability of either case 0? Obviously it's more likely for N to be around 0.2 than around 0.8, but I can't think of how to actually evaluate the difference since they're both exact numbers and there is no region/error given or anything. And the fact that part b) implies that part a) have an ambiguous answer makes me think this might be true. The problem is I'm not really comfortable with probability, so I don't know if my logic is sound here or if I'm missing something.
Any help is greatly appreciated
Best Answer
The density of $Y$, given that $X=0$ is $\text{constant}\cdot e^{-y^2/(2(0.1))}$ (assuming you meant $0.1$ is the the variance, not the standard deviation. Then density of $Y$, given that $X=1$, is $\text{constant}\cdot e^{-(y-1)^2/(2\cdot0.1)}$. So the likelihood function is given by \begin{align} L(x) & = \text{constant}\cdot \left.\begin{cases} e^{-y^2/(2\cdot0.1)} & \text{if }x=0, \\ e^{-(y-1)^2/(2\cdot0.1)} & \text{if }x=1, \end{cases} \right\} \\[10pt] & = \text{constant}\cdot e^{-y^2/(0.2)} \left.\begin{cases} 1 & \text{if }x=0, \\ e^{(2y-1)/0.2} & \text{if } x=1.\end{cases}\right\} \end{align} The expression in front doesn't depend on whether $x=0$ or $x=1$, so it's a "constant" as a function of $x$; it doesn't change as $x$ changes.
To get the posterior distribution, multiply the prior by the likelihood and then normalize (and remember that the prior is uniform in this case): $$ \begin{cases} \dfrac{1}{1+e^{(2y-1)/0.2}} & \text{if }x=0 \\[12pt] \dfrac{e^{(2y-1)/0.2}}{1+e^{(2y-1)/0.2}} & \text{if }x=1 \end{cases} $$ Now plut in $y=0.2$ and you've got it.
One can express this as $$ \operatorname{logit}P(X=1\mid Y=y) = \frac{2y-1}{0.2} = 10y-5, $$ where $$ \operatorname{logit} p = \log_e \frac{p}{1-p}. $$