If $P\left(X\in B\mid\mathcal{A}=\omega\right)=\int_B f_\omega\space d\mu_\omega$ can the $f_\omega$ be chosen in a consistent way so that $f_\omega\left(x\right)$ is measurable in $\omega$?
To put it more precisely, suppose $\left(\Omega_\mathcal{B},\mathcal{B}\right)$, $\left(\Omega_\mathcal{C},\mathcal{C}\right)$ are measurable spaces. Let $\mu,\nu:\mathcal{B}\times\Omega_\mathcal{C}\rightarrow\left[0,1\right]$ be $\sigma$-finite stochastic kernels, i.e. $\mu\left(\cdot,\omega\right),\nu\left(\cdot,\omega\right)$ are $\sigma$-finite probability measures on $\mathcal{B}$ for each $\omega\in\Omega_\mathcal{C}$ and $\mu\left(B,\cdot\right),\nu\left(B,\cdot\right)$ are $\mathcal{C}$-measurable for each $B\in\mathcal{B}$.
Suppose for all $\omega$, $\mu\left(\cdot,\omega\right)\ll\nu\left(\cdot,\omega\right)$, so that the Radon-Nikodym derivative $\frac{d\mu\left(\cdot,\omega\right)}{d\nu\left(\cdot,\omega\right)}$ exists and is unique up to a $\nu\left(\cdot,\omega\right)$-null set. Can a version of this derivative be chosen for each $\omega\in\Omega$, $f\left(x,\omega\right)=\frac{d\mu\left(\cdot,\omega\right)}{d\nu\left(\cdot,\omega\right)}\left(x\right)$, in such a way that $0\leq f$ and $f$ is $\mathcal{B}\otimes\mathcal{C}/\mathfrak{B}$-measurable ($\mathfrak{B}$ being the Borel field on the real line)?
To give the problem a more probabilistic expression, if $\mathcal{C}$ is a sub-$\sigma$-algebras of some probability space $\left(\Omega_\mathcal{A},\mathcal{A},P\right)$ and if $X:\left(\Omega_\mathcal{A},\mathcal{A}\right)\rightarrow\left(\Omega_\mathcal{B},\mathcal{B}\right)$ is some random object that has a regular conditional distribution $\mu=P\left(X\in\cdot\mid\mathcal{C}\right)$, does $X$ have a conditional density $f_{X\mid\mathcal{C}=\omega}(x)$ with respect to $\nu$ ($\mu$ and $\nu$ are assumed to possess all the properties described in the previous paragraphs)? I.e. is there a non-negative, $\mathcal{B}\otimes\mathcal{C}$ -measurable real function $f\left(x,\omega\right)=f_{X\mid\mathcal{C}=\omega}(x)$ such that for all $B\in\mathcal{B}$, $P\left(X\in B\mid\mathcal{C}=\omega\right)=\int_B f_{X\mid\mathcal{C}=\omega}(x)\space\nu\left(dx,\omega\right)$?
Best Answer
Now, a month-and-a-half after i wrote the answer below, i realize the original question is precisely Theorem 58 (p. 52) in Dellacherie, C., and Meyer, P. A. Probabilities and Potential B. North-Holland Publishing Company. 1982 Note that in order for the result to hold, $\left(\Omega_\mathcal{B},\mathcal{B}\right)$ must be separable in the sense of Definition 10 (p. I.10) in Dellacherie & Meyer's volume A, i.e. $\mathcal{B}$ must have a countable generator.
I will prove the claim under the assumptions and definitions described in the last paragraph of the question (i.e. the assumption that $\mu$ is a conditional probability). I will in fact prove a slightly stronger proposition by relaxing the stipulation that $\nu$ be stochastic, requiring only that it be uniformly $\sigma$-finite, i.e. that $\Omega_\mathcal{B}$ can be written as $\bigcup_{n=1}^\infty D_n$ ($D_n\in\mathcal{B}$), where for some positive (finite) constants $k_n$ we have $\nu\left(D_n,\omega\right)\leq k_n$ for all $\omega\in\Omega_\mathcal{C}$. Note that $\mu$ is automatically uniformly $\sigma$-finite by virtue of it being a conditional probability. We can also relax the stipulation that $\mu\left(\cdot,\omega\right)\ll\nu\left(\cdot,\omega\right)$ for all $\omega\in\Omega_\mathcal{C}$, and require only that this condition hold $\left[\mathcal{C},P\right]$-a.s. The proof follows the schema suggested by problem 9 of chapter 1 in [Schervish].
I will define a function $f:\Omega_\mathcal{B}\times\Omega_\mathcal{C}\rightarrow\left[0,\infty\right)$ that is $\mathcal{B}\otimes\mathcal{C}\space/\space\mathfrak{B}$ -measurable ($\mathfrak{B}$ being the standard Borel field on the real line) and such that for all $D\in\mathcal{B}$, $\int_Df\left(x,\omega\right)\space \nu\left(dx,\omega\right)=P\left(X\in D\mid\mathcal{C}=\omega\right)$ $\left[\mathcal{C},P\right]$-a.s. Note that, by Fubini's theorem, if $f$ is non-negative and $\mathcal{B}\otimes\mathcal{C}\space/\space\mathfrak{B}$ -measurable, then for all $\omega\in\Omega_\mathcal{C}$, $f\left(\cdot,\omega\right)$ is non-negative and $\mathcal{B}\space/\space\mathfrak{B}$ -measurable and $\int_D f\left(x,\omega\right)\space \nu\left(dx,\omega\right)$ is non-negative and $\mathcal{C}\space/\space\mathfrak{B}$ -measurable. So in order to show that $\int_Df\left(x,\omega\right)\space \nu\left(dx,\omega\right)=P\left(X\in D\mid\mathcal{C}=\omega\right)$ $\left[\mathcal{C},P\right]$-a.s., it suffices to show that for all $F\in\mathcal{C}$, $\int_F\int_Df\left(x,\omega\right)\space\nu\left(dx,\omega\right)\space P\left(d\omega\right)=P\left(\left\{X\in D\right\}\cap F\right)$.
$f$ will be defined as the Radon-Nikodym derivative of the following couple of measures on $\mathcal{B}\otimes\mathcal{C}$.
a) $Q$, the unique measure that attains the value $Q\left(D\times F\right)=P\left(\left\{X\in D\right\}\cap F\right)$ on every rectangle $D\times F$ with $D\in\mathcal{B}$, $F\in\mathcal{C}$. That such a measure exists and is unique follows from Caratheodory's extension theorem, cf. Theorem 1.53 in [Klenke]. By the formula of total probability, $Q\left(D\times F\right)=\int_F\mu\left(D,\omega\right)\space P\left(d\omega\right)$, so $Q$ is the product measure of $P$ and $\mu$ guaranteed in [Ash & Doleans-Dade]'s Product Measure Theorem (2.6.2) (here's where $\mu$'s uniform $\sigma$-finiteness is used).
b) $\xi$, the product measure of $P$ and $\nu$ guaranteed in the above-mentioned Product Measure Theorem, viz the unique measure that satisfies $\xi\left(D,F\right)=\int_F \nu\left(D,\omega\right)\space P\left(d\omega\right)$ for all $D\in\mathcal{B}$, $F\in\mathcal{C}$ (here's where $\nu$'s uniform $\sigma$-finiteness is used).
In order to be able to define $f=\frac{d Q}{d\xi}$ (the Radon-Nikodym derivative), we should verify that this expression is well defined, namely that $\xi$ is $\sigma$-finite and that $Q\ll\xi$. The $\sigma$-finiteness of $\xi$ is guaranteed by the Product Measure Theorem.
Let's check that $Q\ll\xi$. Let $G\in\mathcal{B}\otimes\mathcal{C}$ s.t. $\xi(G)=0$. We need to check that $Q(G)=0$. By the Product Measure Theorem, $Q(G)=\int_{\Omega_{C}}\mu\left(G^\omega,\omega\right)\space P\left(d\omega\right)$, where $G^\omega$ denotes the section of $G$ at $\omega$: $G^\omega=\left\{\omega'\in\Omega_\mathcal{B}\mid:\space\left(\omega',\omega\right)\in G\right\}$, guaranteed to be $\in\mathcal{B}$. Now, by assumption, $\mu\left(\cdot,\omega\right)\ll\nu\left(\cdot,\omega\right)$ $\left[\mathcal{C},P\right]$-a.s., so we'll be done if we can show that $\nu\left(G^\omega,\omega\right)=0$ $\left[\mathcal{C},P\right]$-a.s. Indeed, by the Product Measure Theorem $\xi(G)=\int_{\Omega_{C}}\nu\left(G^\omega,\omega\right)\space P\left(d\omega\right)$, implying the desired result, since $\xi(G)=0$ and $0\leq \nu$.
Now that we've defined $f$, the second paragraph above indicates that the proof will be complete if we can show that for all $D\in\mathcal{B}$, $F\in\mathcal{C}$, $\int_F\int_Df\left(x,\omega\right)\space \nu\left(dx,\omega\right)\space P\left(d\omega\right)=P\left(\left\{X\in D\right\}\cap F\right)$. But by Fubini's Theorem ([Ash & Doleans-Dade], 2.6.4), by $f$'s definition and by $Q$'s definition, respectively, $\int_F\int_Df\left(x,\omega\right)\space \nu\left(dx,\omega\right)\space P\left(d\omega\right)=\int_{D\times F}f\space d\xi=Q\left(D\times F\right)=P\left(\left\{X\in D\right\}\cap F\right)$.
Q.E.D.
References
Ash, Robert & Doleans-Dade, Catherine. "Probability & Measure Theory". 2000 (2nd edition)
Klenke, Achim. "Probability Theory - A Comprehensive Course". 2008
Schervish, Mark J., "Theory of Statistics", 1995 (1st printing)