Suppose that $\theta \sim N(\mu, \nu^2)$ and $x \, | \, \theta \sim N(\theta, \sigma^2)$.
We can then compute the posterior distribution $ \theta \, | \, x $ using the conjugate prior property of the normal distribution,
$$ \theta \, | \, x \sim N \left(\frac{\nu^2}{\nu^2 + \sigma^2} x + \frac{\sigma^2}{\nu^2 + \sigma^2} \mu, \frac{\nu^2 \sigma^2}{\nu^2 + \sigma^2} \right) $$ and so the conditional expectation $ \mathbb{E}(\theta | x) $ is just the mean of this distribution. I understand this.
What I don't understand is the following. It is then stated that the conditional distribution of the conditional expectation is given by,
$$ \mathbb{E}(\theta \, | \, x) \, | \, \theta \sim N \left(\frac{\nu^2}{\nu^2 + \sigma^2} \theta + \frac{\sigma^2}{\nu^2 + \sigma^2} \mu, \frac{\nu^4 \sigma^2}{(\nu^2 + \sigma^2)^2} \right).$$
I think my problems are two-fold. First, I don't really understand what the conditional distribution of the conditional expectation is conceptually. Shouldn't this just be a function of $\theta$? What is $\mu$ doing in the above equation? Secondly, I have no idea how this can be computed. Could someone explain how this is derived?
Context: this follows from the notes of an economist, shared privately.
Best Answer
You have
$$\operatorname E(\theta \mid x)=\frac{\nu^2}{\nu^2 + \sigma^2} x + \frac{\sigma^2}{\nu^2 + \sigma^2} \mu$$
This is a linear function of the random variable $x$. Since $x$ given $\theta$ is normal, the above also has a normal distribution given $\theta$.
The mean and variance of $\operatorname E(\theta \mid x)$ given $\theta$ are
$$ \operatorname E\left[\operatorname E(\theta \mid x)\mid \theta\right]=\frac{\nu^2}{\nu^2 + \sigma^2} \operatorname E\left[x\mid \theta\right]+\frac{\sigma^2}{\nu^2 + \sigma^2} \mu=\frac{\nu^2}{\nu^2 + \sigma^2} \theta+\frac{\sigma^2}{\nu^2 + \sigma^2} \mu $$
$$ \operatorname{Var}\left[\operatorname E(\theta \mid x)\mid \theta\right]=\left(\frac{\nu^2}{\nu^2 + \sigma^2}\right)^2 \operatorname{Var}(x\mid \theta)=\frac{\nu^4 \sigma^2}{(\nu^2 + \sigma^2)^2} $$