Total variation distance of probaiblity measures

measure-theoryprobability theorytotal-variation

Let $E$ be a set and $\mathcal E\subseteq 2^E$ with $\emptyset\in\mathcal E$. If $\mu:\mathcal E\to\mathbb R$ with $\mu(\emptyset)=0$, then $$\operatorname{Var}_\mu(B):=\sup\left\{\sum_{i=1}^n\left|\mu(B_i)\right|:n\in\mathbb N\text{ and }B_1,\ldots,B_n\in\mathcal B\text{ are disjoint with }\biguplus_{i=1}^nB_i\subseteq B\right\}$$ for $B\subseteq E$. If $E\in\mathcal E$, let $$\left|\mu\right|:=\operatorname{Var}_\mu(E).$$

Assume $(E,\mathcal E)$ is a measurable space and $\mu,\nu$ are probability measures on $(E,\mathcal E)$. Is it possible to show that $$\left|\mu-\nu\right|=\sup_{B\in\mathcal E}\left|\mu(B)-\nu(B)\right\|\tag1$$ or is there a counterexample?

I was able to show the claim assuming that $\mu$ and $\nu$ are both absolutely continuous with respect to a common reference measure. Is it possible to show $(1)$ in general?

Best Answer

You are off by a factor of 2: $\| \mu-\nu\| =2\sup_{B\in\mathcal E}|\mu(B)-\nu(B)|$. For this it's important that $\mu $ and $\nu$ are probabilities, so that $\mu(E)-\nu(E)=0$.

Later addition: Let $\lambda=\mu+\nu$ and let $f$ and $g$ be the Radon-Nikodym densities of $\mu$ and $\nu$ with respect to $\lambda$. It's not hard to check that $\|\mu-\nu\|=\int_E|f-g|\,d\lambda$.

First, because $\mu(E)=\nu(E)=1$, if $B\in\mathcal E$ then, $$ 2|\mu(B)-\nu(B)|=|\mu(B)-\nu(B)|+|\mu(B^c)-\nu(B^c)|\le\operatorname{Var}_{\mu-\nu}(E)=\|\mu-\nu\|. $$ This shows that $\|\mu-\nu\|\ge 2\sup_{B\in\mathcal E}|\mu(B)-\nu(B)|$.

For the reverse inequality, $$ \eqalign{ \|\mu-\nu\|&=\int_E|f-g|\,d\lambda\cr &=\int_{\{f>g\}}(f-g)\,d\lambda+\int_{\{f<g\}}(g-f)\,d\lambda\cr &=(\mu-\nu)(\{f>g\})+(\nu-\mu)(\{f<g\})\cr &=|(\mu-\nu)(\{f>g\})|+|(\mu-\nu)(\{f<g\})|\cr &\le 2\sup_{B\in\mathcal E}|\mu(B)-\nu(B)|. } $$

Related Question