Independence and Conditional Probability Density Function

conditional probabilitydensity functiongamma distributionindependenceprobability

Let us assume that $n \in \mathbb{N}$ and that $a_1 , a_2 , c_1 , c_2, d_1, d_2$ are positive real numbers. Let us define random variables

$$ \theta \sim \Gamma (a_1, b_1 ) , \ \lambda \sim \Gamma ( a_2 , b_2 ), \ k \sim U( \{ 1 ,\dots , n \} ) ,$$

where

$$ b_1 \sim \Gamma ( c_1 , d_1) \text{ and } b_2 \sim \Gamma (c_2 , d_2 ). $$

Let us also assume that $\theta , \lambda $ and $k$ are independent. Moreover, let $b_1$ and $b_2$ be independent. Next, we see that

$$ p_{\lambda, \theta, b_1 , b_2 , k} ( \alpha , \beta , \gamma , \omega , \tau ) = p_{\lambda , \theta , k| b_1 , b_2} (\alpha , \beta , \tau | \gamma , \omega ) \cdot p_{b_1, b_2} ( \gamma , \omega ). $$

Since $b_1$ and $b_2$ are independent, then $p_{b_1, b_2} ( \gamma , \omega ) = p_{b_1} ( \gamma) \cdot p_{b_2} ( \omega )$. My question is the following. Does the independence of $\theta, \lambda$ and $k$ imply that

$$ p_{\lambda , \theta , k| b_1 , b_2} (\alpha , \beta , \tau | \gamma , \omega ) = p_{\lambda | b_1 , b_2} ( \alpha | \omega , \gamma) \cdot p_{\theta | b_1 , b_2} ( \beta | \gamma , \omega ) \cdot p_{k| b_1 ,b_2} ( \tau | \gamma ,\omega ). $$

In other words, is it true that

$$ p_{\lambda, \theta, b_1 , b_2 , k} ( \alpha , \beta , \gamma , \omega , \tau ) = p_{\lambda | b_1 , b_2} ( \alpha | \omega , \gamma) \cdot p_{\theta | b_1 , b_2} ( \beta | \gamma , \omega ) \cdot p_{k| b_1 ,b_2} ( \tau | \gamma ,\omega ) \cdot p_{b_1} ( \gamma) \cdot p_{b_2} ( \omega ). $$

Symbol $p_{\textbf{X}}$ denotes the probability density function of a random vector $\textbf{X}$.

Update

To be more precise, the joint probability density function I had in mind is the following.

$$ \bigg( \frac{ \omega^{a_2} }{\Gamma (a_2)} \alpha^{a_2 -1} e^{- \alpha \omega} \bigg) \ \bigg( \frac{ \gamma^{a_1} }{\Gamma (a_1)} \beta^{a_1 -1} e^{- \beta \gamma} \bigg) \frac{1}{n} \ \bigg( \frac{ d_1^{c_1} }{\Gamma (c_1)} \gamma^{c_1 -1} e^{- \gamma d_1} \bigg) \ \bigg( \frac{ d_2^{c_2} }{\Gamma (c_2)} \omega^{c_2 -1} e^{- d_2 \omega} \bigg)$$

Best Answer

In the language of graphical models you wrote a pdf corresponding to the following graph:

enter image description here

but the path $\theta \rightarrow b1 \rightarrow \lambda$ is not blocked, therefore $\theta$ and $\lambda$ are in general not independent (check the concept of $D-$separation).

In other words I think that the pdf you wrote is in general incompatible with even the independence between these two variables.

As a check, try to marginalize your pdf over $\lambda,b1,b2$ and try to write it as a product (you should not be able to do it).

UPDATE: To try to answer the comment, we can try to check what graphical models are compatible with the hypothesis. We start from the graphical form with only the arrows from b1 to $\theta$ and from b2 to $\lambda$. Now you start adding arrrows with the constraint that:

  • you cannot have arrows between $b1$ and $b2$
  • you cannot have arrows between $\theta,\lambda$ and $k$
  • you should not generate unblocked paths between $b1$ and $b2$
  • you should not generate unblocked paths between $\theta,\lambda$ and $k$

By making som trials it looks to me that you cannot insert any other arrow. This would lead to the distribution:

$$p \sim p(\theta|b1)p(\lambda|b2)p(b1)p(b2)p(k)$$

as the only one representable in the graphical model respecting the conditions. Problem is, I do not know/remember if all p.d.f. should be representable in a graphical model like this.

Related Question