Fix a measurable space $(X,\Sigma)$. Let $\mu:\Sigma\to\mathbb{R}$ be some signed measure. Then the total variation norm is defined by
$$
\|\mu\|_{\textrm{TV}}=\mu_{+}(X)+\mu_{-}(X),\quad (*)
$$
where $\mu=\mu_{+}-\mu_{-}$ is the Hahn decomposition of $\mu$. Now, let $\mu,\nu:\Sigma\to[0,1]$ be two probability measures. Then
$$
\|\mu-\nu\|_{\textrm{TV}}=2\max_{A\in\Sigma}(\mu(A)-\nu(A)),\quad (**)
$$
To see this, recall that the Hahn decomposition
$$
\mu-\nu=(\mu-\nu)_{+}-(\mu-\nu)_{-}
$$
has the form
$$
(\mu-\nu)_{\pm}(A)=(\mu-\nu)(A\cap A_{\pm}),\quad A\in\Sigma,
$$
where $A_{+}\cup A_{-}=X$, and the choice of the sets $A_{+}$, $A_{-}$ is unique up to a set of zero measures
$$
|\mu-\nu|=(\mu-\nu)_{+}+(\mu-\nu)_{-}.
$$
This apparently immediataly implies (**) with the maximum attained at the set $A_{+}$.
My question: I'm having a hard time connecting the definition of total variation norm in (*) with the arguments that lead to (**). Any help is greatly appreciated.
Best Answer
From the Hahn decomposition theorem, you can check that the definition
$$ \|\mu\|_{\textrm{TV}}=\mu_{+}(X)+\mu_{-}(X),\quad (*) $$
can be rewritten as
$$ \|\mu\|_{\textrm{TV}}=\max_{A \in \Sigma}\mu(A)-\min_{A \in \Sigma}\mu(A) $$
with the max and min achieved at $A_+$ and $A_-$ respectively.
So for the difference of two probability measures,
\begin{align*} \|\mu-\nu\|_{\textrm{TV}}&=\max_{A \in \Sigma}(\mu(A)-\nu(A))-\min_{A \in \Sigma}(\mu(A)-\nu(A))\\ &=(\mu(A_+) - \nu(A_+)) - (\mu(A_-)-\nu(A_-))\\ &=(\mu(A_+) - \nu(A_+)) - ((1-\mu(A_+))-(1-\nu(A_+)))\\ &=2(\mu(A_+) - \nu(A_+))\\ &=2\max_{A \in \Sigma}(\mu(A)-\nu(A)) \quad (**) \end{align*}