I've been trying to look this up, but I must not be searching correctly because I think it's a pretty basic question.
What I want to know is what do subscripts mean when talking about the probability (or expectation) of something? For example $P_{\theta}(x|y^{(i)})$. What does the $\theta$ subscript mean here?
I saw someone ask the same question here, but I don't understand the answer.
Best Answer
Maybe this helps:
$P$ is the distribution of random variable $x$ given the value of random variable $y$. And this distribution has parameters $\theta$. By varying the parameters, you get different distributions. For example, probability distribution over a random variable $x$ with uniform distribution on support $[a,b]$ can be written as $$P_{(a,b)}(x) = \frac{1}{b-a}I_{[a,b]}(x).$$ You will get a different distribution by varying $a$ and $b$.
Edit: Frequently, you will see the probability density function of a random variable $X$ denoted as $f_X(x)$, here the subscript is just used to remember that $f_X$ is a probability density of $X$. Here $X$ is not a parameter. But if is written as $f_\theta(x)$, then $\theta$ is likely to be a parameter of the density function of the random variables $X$ which can also be written as $f_X(x|\theta).$ You will have to see what it is by the context which is very easy to see in all the cases I have encountered.