Continuous functions in strong(norm) topolgy and weak topology

functional-analysisgeneral-topologynormed-spacesweak-topology

While reading Wasserstein GAN paper and in Appendix A, it says that

The norm topology is very strong. Therefore, we can expect that not many functions $\theta \mapsto \mathbb{P}_\theta$ will be continuous when measuring distances between distributions with $\delta$

From what I understand, strong topology has more open sets than weak topology, hence I assumed that a continuous function on weak topology implies continuity in strong topology but not vice versa. Hence there would be more continuous functions in strong topology than weak topology. However, several other posts have shown that continuity in strong topology implies continuity in weak topology here. So my questions are:

  1. does continuity in strong topology imply continuity in weak topology?
  2. does continuity in weak topology imply continuity in strong topology?
  3. How do we know that weak topology has more continuous functions than strong topology?

Best Answer

It seems that what the authors of the paper are saying is that the topology if total variation in the space of finite measures is to restrictive. The weak topology they are referring there is the weak topology $\sigma(\mathcal{M},\mathcal{C}_b(X))$ where $\mathcal{M}$ is the space of finite measures, and $\mathcal{C}_b(X)$ is the space of continuous bounded functions (on $X$). A local base of the topology $\sigma(\mathcal{M}(X),\mathcal{C}_b(X))$ is given by sets of the form $$V(f_1,\ldots,f_n;\varepsilon)=\{\mu\in\mathcal{M}:|\mu(f_j)|<\varepsilon\}$$ where $\mu(f_j)=\int_Xf_j\,d\mu$, $f_1,\ldots,f_n\in\mathcal{C}_b(X)$, $n\in\mathbb{N}$. In this topology, a net $(\mu_\alpha:\alpha \in D)$ converges to $\mu$ iff for any $f\in\mathcal{C}_b(X)$ $$\lim_\alpha \mu_\alpha(f)=\mu(a)$$

I assume for the rest of this posting that $X$ is a complete separable metric space.

As for your questions:

  1. If $\mu_n$ converges to $\mu$ in total variation, then $\mu_n$ converges to $\mu$ in $\sigma(\mathcal{M}(X),\mathcal{C}_b(X))$. Indeed, for $f\in\mathcal{C}_b(X)$ $$\Big|\int_Xf\,d\mu_n-\int_Xf\,d\mu\Big|\leq\int_X|f|\,d|\mu_n-\mu|\leq\|f\|_u\|\mu_n-\mu\|_{TV}\xrightarrow{n\rightarrow\infty}0$$

  2. Consider a sequence $x_n\in X\setminus\{x\}$ such that $x_n\xrightarrow x$ in $X$. Consider the measures $\delta_{x_n}$ and $\delta_x$ where $\delta_{x_n}(A)=\mathbb{1}_{A}(x_n)$ (similar for $\delta_x$). You can easily check that $\mu_n$ converses weakly to $\mu$ (i.e. in the $\sigma(\mathcal{M}(X),\mathcal{C}_b(X))$ topology); however, $\|\delta_{x_n} -\delta_x\|_{TV}=1$

  3. I leave this for the OP to think about.