CLARIFICATION:
If someone could please help me understand the following: When examining the expected value in this specific situation, how is the distribution of $\theta$ relevant? What difference would it make whether $\theta$ is $10^{10}$ or $10^{-10}$, $-3$, or normal or not?
How does one take a partial derivative with respect to a random variable?
For an assignment, I need to find $E_{Y\mid\theta} \left[- \dfrac{\partial^2}{\partial^2\theta} \ln[P(y\mid\theta)] \right]$, where
$Y \sim N(\theta, 1)$ and $\theta\sim N(0, \delta^2)$, $\delta$ is known
And I realized I do not know how to compute $\dfrac{\partial^2}{\partial^2\theta} \ln[P(y\mid\theta)]$
If I treat $\theta$ as a standard variable and take the second partial derivative of $\ln[P(y\mid\theta)]$ I get a value of $-1$. This does not make sense to me as a correct solution as I would get the same result regardless of $\theta$'s distribution.
I am bit confused and any clarification or assistance would be appreciated.
Thank you.
Best Answer
I dislike using the same notation for both a random variable and the argument to its density function or cumulative distribution function. Thus $F_X(x)=\Pr(X\le x)$, and the meanings of $X$ and $x$ are different. The lower-case $x$ can be any number, so for example $F_X(3)=\Pr(X\le3)$, etc. I think a good case can be made that certain notational usages ultimately make clear thinking easier. In particular, $f_Y(y)$ is not a random variable, but $f_Y(Y)$ is, and it looks as if that's what we've got here. So let's say $$ Y\mid\theta \sim N(\theta,1)\text{ and }\theta\sim N(0,\delta^2). $$
I take it $P(y\mid\theta)$ is supposed to be the conditional density of $Y$ given $\theta$ evaluated at $y$. I'll write $f_{Y\mid\theta}(y)$.
Everything you're asking about is conditional on the value of $\theta$ (or did I miss something?) so the distribution of $\theta$ doesn't matter at all; for the purposes of this problem it's not even a random variable. I don't know what a partial derivative with respect to a random variable would mean (except maybe a Radon--Nikodym derivative, but that has no relevance here).
Now we have $$ \mathbb E\left[ -\frac{\partial^2}{\partial\theta^2} \ln f_{Y\mid \theta}(Y) \mid\theta \right]. $$ $$ = \mathbb E \left[ -\frac{\partial^2}{\partial\theta^2} \ln\left( \frac{1}{\sqrt{2\pi}} \exp\left( \frac{-(Y-\theta)^2}{2} \right) \right) \right] =\mathbb E\left[ -\frac{\partial^2}{\partial\theta^2}\left( \frac{-(Y-\theta)^2}{2} \right) \right]. $$ Then it appears you have the expected value of a constant.