Suppose we have random variables $X$ and $X_n$, $n = 1, 2, …$, on a countable sample space $\Omega$, probability measure $\mathbb{P}$ and $\sigma$-algebra $\mathcal{F}$ , such that $X_n \stackrel{p}{\rightarrow} X$ (i.e. $\mathbb{P}(|X_n – X| > \epsilon) \rightarrow 0$). Does it necessarily hold that $X_n$ converges to $X$ in total variation distance, i.e. $\sup_{A \in \mathcal{F}} |X_n (A) – X(A)| \rightarrow 0$?
I suspect that this is the case, at least for a countable $\Omega$. When $\Omega$ is countable, we want to show the following,
$$
\sup_{A \in \mathcal{F}} |X_n (A) – X(A)| = \frac{1}{2} \sum_{\omega \in \Omega} |X_n(\omega) – X(\omega)| \rightarrow 0.
$$
First, construct $A_1 \subset A_2 \subset … \subset \Omega$, such that for any $k < \infty$, $|A_k| < \infty$ and $\cup_i A_i = \Omega$. For any $\epsilon > 0$, choose $\epsilon_k' = \epsilon/|A_k|$. Then,
$$
\frac{1}{2} \sum_{\omega \in A_k} |X_n(\omega) – X(\omega)| \leq \frac{1}{2} \sum_{\omega \in A_k} \epsilon_k' + (|X_n(\omega) – X(\omega)| – \epsilon_k ') 1_{\{ |X_n(\omega) – X(\omega)| > \epsilon_k'\}}.
$$
By convergence in probability, hence choose $n$ large enough such that the indicator is $0$ a.s. , and now, the above is bounded by $\epsilon$. Since $\epsilon$ is arbitrary, and by induction, we can show that the total variation converges to $0$.
But of course, this isn't quite a concrete proof and might even be incorrect. I suspect the last part about the indicator is $0$ isnt correct. Is this the case?
Best Answer
This is not true. Note that, one can prove that on a countable probability space the convergence in probability is equivalent to convergence almost surely, e.g. see here.
Now regarding your first question: Your formula for the total variation distance is not right. You seem to confusing the random variables and the probability measure. Still, working with your quantities we had the following counterexample.
Let $\Omega = \Bbb N$,$\Bbb P \sim \text{Geo} (1/2)$ ($\Bbb P (\{\omega\}) = (1/2)^{\omega}$), $X = 0$, and $X_n (\omega ) = 1_{ \{n\}} (\omega)$.
Then we have
$$\frac 1 2 \sum_{\omega\in\Omega} \vert X_n ( \omega ) - X(\omega) \vert = \frac 1 2$$ for every $n\in\Bbb N$, but also for every $1 >\varepsilon > 0$ that $$\{\vert X_n - X \vert > \varepsilon \} = \{n\},$$ which implies $$\Bbb P (\vert X_n - X \vert > \varepsilon ) = \Bbb P (\{n\}) = \left(\frac 1 2 \right) ^n \to 0$$ as $n \to \infty$.
You can have the "total variation" distance between the random variables, but it is more a distance between their distributions.
The definition of the total variation would be
$$\sup_{A\in\mathcal F} \vert \Bbb P (X\in A ) - \Bbb P (X_n \in A) \vert$$
Then the negative answer to your question does not really depend on the probability space $\Omega$. Regardless of $\Omega$ any random variables with distributions $X_n \sim \delta_{1/n}$ will converge to $X=0$ in probability but not in total variation since:
$$\sup_{A\in\mathcal F} \vert \Bbb P (X\in A ) - \Bbb P (X_n \in A) \vert = \vert \Bbb P (X\in \{0\} ) - \Bbb P (X_n \in \{0\}) \vert = \vert 1 - 0\vert = 1$$