Let $\mathcal{B}$ be the set of all Borel sets in $\mathbb{R}$. Then the total variation distance between two probability measures $\mu$ and $\nu$ on $\mathbb{R}$ is,
$$
TV(\mu,\nu)=\sup_{A\in \mathcal{B}}|\mu(A)-\nu(A)|
$$
Let $S_n=\frac{X_1+\cdots+X_n}{\sqrt{n}}$, where $X_{i}=\pm 1,$ iid $,\forall i=\{1,\cdots,n\}$ and $Z$ is the standard normal random variable. In these notes, on page 6, it is claimed that $TV(S_n,Z)=1,\forall n$.
My proof for this is that since the sample space, say $A$, of $S_n$ is countable and discrete for a given $n$. Then $\mu(A)=1$ and $\nu(A)=0$ since $Z$ is a continuous distribution. Is this correct?
If it is, wouldn't the total variation distance between two continuous probability measures be always one?
Best Answer
Your argument is for one discrete and one continuous r.v. so you cannot conclude that the total variation distance between any two continuous distributions is $1$. (What if the two are the same?)
We always have $|\mu (A)-\nu (A)| \leq 1$ and there is a countable set $A$ with $P(S_n \in A)=1$. So $TV(S_n,Z) \geq |\mu (A)-\nu (A)|=|1-0|=1$.
The total variation distance between a discrete and a continuous distributions is $1$.