I'm trying to generalise a formula that takes the mean of some variances to it work with vectors.
I'm not sure it makes sense to take the variance between a bunch of vectors, rather it is more suited to generate a covariance matrix.
So, is there a way to take the mean of a bunch of covariance matrices?
Best Answer
If the original covariance matrices are of the same dimension, i.e., you have $n\times n$-matrices $\Sigma_1,\ldots,\Sigma_N$, you could just take the sample mean: \begin{equation} \bar{\Sigma} = \frac{1}{N}\left(\Sigma_1 +\ldots+\Sigma_N \right), \end{equation} which corresponds to taking elementwise means of the elements of the covariance matrices, i.e., the $i,j$th element will be \begin{equation} \bar{\Sigma}_{i,j} = \frac{\Sigma_{1(i,j)}+\Sigma_{2(i,j)}+\ldots+\Sigma_{N(i,j)}}{N}. \end{equation}
Provided that the original covariance matrices are valid (positive-semidefinite), this will produce a valid covariance matrix, as positive-semidefiniteness is preserved under summing and multiplying by a positive constant ($1/N$).
Whether taking this mean is reasonable will depend on the context, which is not explained in the question.