If I have a joint probability density $f(x,y)$, are there any regularity conditions that need to be met in order to allow $f(x,y) = f(y|x)f(x)$, or is this always possible? The statistics books I consulted state this as something obviously true without elaborating further.
[Abramovich & Ritov, p. 186] Given the joint distribution of $X$ and $Y$ one can always obtain their marginal distributions. Thus, for the continuous case, if $f_{XY}(x,y)$ is the joint density of $X$ and $Y$, their marginal densities $f_X(x)$ and $f_Y(y)$ are $f_X(x) = \int_{-\infty}^{\infty} f_{XY}(x,y)dy$ and $f_Y(y) = \int_{-\infty}^{\infty} f_{XY}(x,y)dx$. Two random variables X and Y are called independent if their joint density is a product of the marginal densities $f_{XY}(x,y) = f_X(x) \cdot f_Y(y)$ …
[Spanos, p. 139]: It should come as no surprise to learn that from the joint distribution one can always recover the marginal (univariate) distributions of the individual random variables involved.
I just wondered if there's more to it.
And, a related question: could there be a situation where $f(x,y) = f(y|x)f(x)$ is possible, but $f(x,y) = f(x|y)f(y)$ is not?
Best Answer
As is typically abused in Statistics, I write $f(x,y), f(x \mid y), f(x),$ etc. for the densities of $(X,Y), X \mid Y = y, X,$ etc. I assume all of these are continuous.
You start with $f(x,y).$ Given this, you construct $f(y) = \int dx\ f(x,y)$ and then $f(x \mid y) = \frac{f(x,y)}{f(y)}.$ Note that if $f(x,y) = 0$ on the line $\{x\} \times \mathbf{R}$ then $f(y) = 0$ and reciprocally, if $f(y) = 0,$ then $f(x,y) = 0$ for all $x$ (since we are assuming $f$ is continuous). Then, by definition, $f(x,y) = f(y) f(x \mid y)$ (when $f(x,y) = f(y) = 0,$ the value of $f(x \mid y)$ can be arbitrarily defined as any number). So, you are asking whether or not $f(x,y) = f(x \mid y) f(y);$ the answer is yes, always, by definition.
Addendum: the question as to why $f(x \mid y)$ is a density for $X \mid Y = y$ is somewhat difficult and usually out of scope in statistics books. It has to do with the definition of conditional expectation and regular conditional probabilities.