I've just come across the sentence:
"Suppose you have the sample $\ldots$ from a normal distribution with mean $\mu$ and variance unity."
What does variance unity mean? Variance equals 1? I can't seem to find clarification of the phrase.
[Math] What does “variance unity” mean?: “A normal distribution with mean $\mu$ and variance unity”.
probabilityvariance
Related Solutions
For this you start by writing out your loglikelihood $$ \begin{align} \log p(y|\mu,\sigma) &= \sum_i \log (y_i | \mu,\sigma) \\ &= \sum_i \left(-\frac{(y_i-\mu)^2}{2\sigma^2} - \log(\sqrt{2\pi\sigma^2}) \right)\\ &= -n\log \sigma - \frac{1}{2\sigma^2}\sum_i(y_i-\mu)^2 + \mbox{constant}. \end{align} $$ Now what happens next is the following decomposition of the sum of squares term $$ \begin{align} \sum_i (y_i - \mu)^2 &= \sum_i \left( y_i - \bar{y} + \bar{y} - \mu)\right)^2 \\ &= \sum_i \left((y_i-\bar{y})-(\mu-\bar{y})\right)^2, \end{align} $$ you might like to go from here?
Ok if you are happy with that rearrangement of the likelihood then looking at the Bayesian aspects we are looking for the posterior $$ p(\mu,\log \sigma | y) \propto p(y|\mu,\log \sigma) \pi(\mu,\log \sigma) $$ where $\pi(\cdot)$ is our prior, now this is assumed to be uniform which will also sometimes be given like $$ \pi(\log \sigma) \propto 1, $$ now in specifying a uniform prior there is a little bit of handwaving going on - this is an improper prior if I want $\log \sigma$ to have an infinite support - I'm sure Gelman discusses this more in the book, a few releavent points are that improper priors can still lead to proper posteriors and we can compactify the infinite support if we want. Anyway assuming the $\mu$ also has an (independently of $\log\sigma$) uniform prior then taking the log of the posterior I have $$ \begin{align} \log p(\mu,\log \sigma | y) &\propto \log p(y | \mu,\log \sigma ) + \log \pi(\log \sigma) + \log \pi(\mu) \\ &\propto \log p(y|\mu,\log \sigma) \end{align} $$ Where to get the second line we used that both priors are proportional to a constant. Now as I mentioned as it stands this $\log \sigma$ on the righthand side is just a reparameterisation of the usual log-likelihood term which as far as writing this term out makes no difference, where it *will* make a difference is when we proceed to carry out differentiation to construct an approximation and so we will be differentiating with respect to $\log \sigma$ and not $\sigma$.
Finally why carry out this change of variables? Well I suspect what the author is going to do is construct a Laplacian approximation which amounts to a Gaussian approximation around the posterior mode, now if we were to construct this in the original $(\mu,\sigma)$ then the Guassian has support $\mathbb{R}^2$ and so will give positive probability to negative variances.
What is below is for $\mu=0$ (and variance renamed $\sigma^2$). Then $\mathbb{E}[\sin X]=0$, and you have $$ \operatorname{Var} \sin X = \mathbb{E}[\sin^2 X] = \frac{1}{2}\left(1-\mathbb{E}[\cos 2X]\right) $$ and $$ \mathbb{E}[\cos 2X] = \sum_{k=0}^\infty (-1)^k\frac{2^{2k}}{(2k)!} \mathbb{E}[X^{2k}] = \sum_{k=0}^\infty (-1)^k\frac{2^{2k}}{(2k)!} \sigma^{2k} (2k-1)!! = \sum_{k=0}^\infty (-1)^k \frac{2^{k}\sigma^{2k}}{k!} = e^{-2\sigma^{2}} $$ and therefore $$ \operatorname{Var} \sin X = \boxed{\frac{1-e^{-2\sigma^2}}{2}} $$ You can deal with the variance of $\cos X$ in a similar fashion (but you now have to substract a non-zero $\mathbb{E}[\cos X]^2$), especially recalling that $\mathbb{E}[\cos^2 X] = 1- \mathbb{E}[\sin^2 X]$.
Now, for non-zero mean $\mu$, you have $$ \sin(X-\mu) = \sin X\cos \mu - \cos X\sin\mu $$ (and similarly for $\cos(X-\mu)$) Since $X-\mu$ is a zero-mean Gaussian with variance $\sigma^2$, we have computed the mean and variance of $\sin(X-\mu)$, $\cos(X-\mu)$ already. You can use this with the above trigonometric identities to find those of $\cos X$ and $\sin X$. (it's a bit cumbersome, but not too hard.)
Without knowing anything about the distribution of $X$, I don't think there's much you can do.
Best Answer
(I'm putting this as an answer so that the question does not remain unanswered.)
Variance unity means a variance equal to 1.