Let $\mathbf{x}\in\Bbb{R}^n$ be an $n$-dimensional real vector distributed normally with mean vector $\mu\in\Bbb{R}^n$ and covariance matrix $\Sigma$; i.e. $\mathbf{x}\sim\mathcal{N}(\mu,\Sigma)$.
The Mahalanobis distance of $\mathbf{x}$ is defined as
$$
D_M(\mathbf{x}) = \sqrt{ (\mathbf{x}-\mu)^\top\Sigma^{-1}(\mathbf{x}-\mu) }.
$$
I am interested in defining a distance between two $n$-dimensional random vectors $\mathbf{x}\sim\mathcal{N}(\mu_x,\Sigma_x)$ and $\mathbf{y}\sim\mathcal{N}(\mu_y,\Sigma_y)$ in a similar way as above.
The Bhattacharyya distance between two multivariate distributions $\mathcal{N}(\mu_x,\Sigma_x)$ and $\mathcal{N}(\mu_y,\Sigma_y)$ includes the following "Mahalanobis term" (as Wikipedia suggests):
$$
d(\mathbf{x},\mathbf{y})=\sqrt{(\mathbf{x}-\mathbf{y})^\top \Bigg(\frac{\Sigma_x+\Sigma_y}{2}\Bigg)^{-1}(\mathbf{x}-\mathbf{y})}.
$$
Would that be a good candidate?
Best Answer
Yes, that is a fine candidate, because in the first case you are basically computing the distance to the mean, but for two distributions you have two actual sample vectors you can compute the distance between, so there's really no reason to include the means in the distance calculation. But if you're interested in variants, the basic issue is how to define the matrix in the middle. If $x$ and $y$ were sampled from distributions with different covariances $\Sigma_x$ and $\Sigma_y$, then one natural thing to do is to take the average of the two covariances, as you have found. At the very least, this is consistent in the sense that you get $\Sigma_x^{-1}$ as the inverse matrix if $\Sigma_x = \Sigma_y$. However there are possibly variants which may, for example, take the average of the inverses rather than the inverse of the average. This also has the consistency property when $\Sigma_x = \Sigma_y$. I would have to read more to explain perhaps why they chose the average of the covariances (before taking inverse) over all other choices