How do we define independence for random vectors?
Eg, let $\mathbf{e}$ be a random n by 1 vector and $\mathbf{b}$ be a random k by 1 vector (where n does not necessarily have to equal to k), then how do we define independence between $\mathbf{e}$ and $\mathbf{b}$?
For example, if we take just 2 random variables, X, Y then X and Y are independent iff f(x,y) = f(x)f(y) where f(x,y) characterizes their joint pdf and f(x), f(y) are their marginal pdfs. If we adapt this onto the random vector case, we have $f(\mathbf{e}, \mathbf{b}) = f(\mathbf{e}) f(\mathbf{b})$, examining the RHS, we see that $f(\mathbf{e})$ is simply the joint distribution of $e_1, e_2, …, e_n$ and likewise, $f(\mathbf{b})$ is the joint distribution of $b_1, b_2, …, b_k$, but then what is $f(\mathbf{e}, \mathbf{b}) $? How is it defined?
Continuing on, for a more special case, consider when $\mathbf{e}$ is jointly multivariate normal, ie, $\mathbf{e} \sim N(\mathbf{\mu_e}, \mathbf{\Sigma_e})$ and $\mathbf{b}$ is also jointly multivariate normal, ie, $\mathbf{b} \sim N(\mathbf{\mu_b}, \mathbf{\Sigma_b})$. Now in the special case of when two jointly normally distributed variables X and Y, a sufficient condition for independence is when cov(X,Y) = 0 where cov(.) denotes the covariance between X and Y. How then, do we generalise that into the random vector case? Ie, can we say that $\mathbf{e}$ and $\mathbf{b}$ are independent iff $cov(\mathbf{e}, \mathbf{b}) = 0$? But then we would have to show $\mathbf{e}$ and $\mathbf{b}$ are jointly normally distributed… but this doesn't really make sense since $\mathbf{e}$ and $\mathbf{b}$ are already itself jointly normally distributed, so then wouldn't we be talking about the 'joint' distribution of two joint distributions? (How would I make sense of that?)
Thanks
Best Answer
The best definition for independence I can give you, without sweating measure theory all that much, is this:
Two random variables $X$ and $Y$ (which share the same probability space, but we don't assume have the same codomain) are independent if the following holds:
For any event $A$ for $X$ and any event $B$ for $Y$, we have $$ P(X\in A,\ Y\in B)=P(X\in A)\cdot P(Y\in B). $$ If you want to start talking about probability densities and things of that nature for the case when you have two random vectors of different dimensions, you're going to need some more serious theory; such things can be defined fairly easily in terms of measure theory, but not as easily otherwise.
Of course, if you have an $n$-long vector and a $k$-long vector, you can think of their joint distribution as being an $(n+k)$-long vector; in this case, assuming you are talking about continuous distributions, you could probably think of it as the condition that $$ f_{\vec{x},\vec{y}}(x_1,\ldots,x_n,y_1,\ldots,y_k)=f_{\vec {x}}(x_1,\ldots,x_n)\cdot f_{\vec{y}}(y_1,\ldots,y_k), $$ where $f_{\vec{x},\vec{y}}$ is the joint density function and $f_{\vec{x}}$ and $f_{\vec{y}}$ are their respective marginal densities.