I have a problem with the intuition of the conditional probability.
Suppose we have a multivariate normal distribution (bivariate for simplicity) with mean $\mu$ and covariance matrix $\Sigma$ with the following form.
I undertand that the intuitive idea of conditional probability is to fix one of the dimensions to certain value doing what would be a "slice" to the multivariate Gaussian distribution, something like this. Whose result is another Gaussian distribution with one dimension less, mean $\mu'$ and covariance matrix $\Sigma'$.
Seeing that figure, intuition tells me that the conditioned probability function should vary depending on the fixed point, that is to say, higher and peaked when the fixed point is near the mean $\mu$ and flatter at the extremes. But what I find is that only the mean is affected, the variance is constant regardless of the point. See
this and this.
How can this be possible, is there something I am misunderstanding?
Thank you in advance!
Best Answer
Note that your conditional distribution of $Y$ given $X$ is something of the form $N(b_0 + b_1x, \sigma^2)$. Namely, by fixing $X$, your mean is its affine transformation $b_0 + b_1x$ to the symmetry axis of the ellipse. That is, you always "slice" the ellipse at this axis. Regarding the variance, note that if your intuition is correct then the "sliced" bell, i.e., the curve at the extremes is not only "flatter" but also the area under the curve should be less than one. That contradicts the fact the conditional distribution is proper random variable. Hence, in the extremes you need the "add up" the missing area in order to get a proper density function, this "correction" reconstructs the "missing" variance.