EDIT:
For the Bounty, I made a substantial edit revision concerning the structure of the question, to make it more readable (hopefully). Moreover I added a question on problem 2.7 of Billingsley’s book.
I have two problems concerning weak convergence of probability measures in product spaces, that arose from Billingsley’s classic “Convergence of Probability Measures” (Chapter I, Section 2, Subsection “Product Spaces”).
Below there are the part of the book that made me wonder, the numbered questions I have, plus my thoughts (hidden, in order to lighten the overall immediate reading of the question).
Notation: $P$ is a probability measure on $T = S' \times S''$, $P_n \Rightarrow P$ denotes weak convergence, while a $P$-continuity set is the family of those sets $A \in \mathcal{S}$ such that $P(\partial A)= 0$, where $\mathcal{S}$ denotes the Borel $\sigma$-algebra of $S$, and $\partial A$ denotes the boundary of $A$. Concerning product spaces, $\mathcal{T} := \mathcal{S}' \times \mathcal{S}''$ is the $\sigma$-algebra of $T$, while $P’$ is the marginal distribution on $\mathcal{S}’$ of $P$ on $\mathcal{T}$, defined as $P’ (A) := P(A \times S’’)$ (and the same applies to $P’’$).
I PART
The first problem is with the following Billingsley’s statement in bold.
[Theorem 2.8.ii. should be a sort of a partial converse of the – trivial – following proposition, here named in the following way:
Proposition 1: If $P_n \Rightarrow P$, then $P^{'}_n \Rightarrow P'$, and $P^{''}_n \Rightarrow P’’$.]
Therefore, we have the following theorem, in which (ii) is an obvious consequence of (i).
Theorem 2.8.
(i) If $T = S' \times S''$ is separable, then $P_n \Rightarrow P$ if and only if $P_ n (A’ \times A’’) \to P(A' \times A'')$ for each $P’$-continuity set $A’$ and each $P’’$-continuity set $A''$.
(ii) If $T$ is separable, then $P^{'}_n \times P^{''}_n \Rightarrow P’ \times P’’$ if and only if $P^{'}_n \Rightarrow P'$ and $P^{''}_n \Rightarrow P’’$.
II PART
The Second problem concerns problem 2.7 in the book, that reads:
Problem 2.7: The uniform distribution on the unit square and the uniform distribution on its diagonal have identical marginal distributions. Relate this to Theorem 2.8.
Questions:
-
How do we prove (ii)?
1.a. How do we prove the ($\Leftarrow$) direction of (ii)?
1.b. How do we actually use (i) to prove (ii), as Billingsley suggests?
1.c. Is my way of addressing the problem below sound? -
How do we actually relate problem 2.7 to theorem 2.8?
2.a. Do we actually have to use the setting in the problem to come up with some measure that works as a counterexample of Proposition 1, or
2.b. It is enought to notice that the weak limit is now not unique, and hence the converse of proposition 1 does not hold anymore?
Here my thoughts on question 1:
Attempted proof of (ii):
(only if) Assume that $P^{'}_n \times P^{''}_n \Rightarrow P' \times P’’$. Thus, by taking the marginals $P'$, and $P''$ on $P^{'}_n \times P^{''}_n \Rightarrow P' \times P''$, we obtain that $P{'}_n \Rightarrow P’$, and $P{''}_n \Rightarrow P''$.
(if) Assume that $P^{'}_n \Rightarrow P'$, and $P^{''}_n \Rightarrow P''$. Let $A' \in \mathcal{S}'$, $A'' \in \mathcal{S}'$ be arbitrary, such that $A'$ is a $P'$-continuity set, and $A''$ is a $P''$-continuity set. [… and here it ends. I thought that we could use the Portmanteau theorem to get somewhere from the fact that $P^{'}_n \Rightarrow P'$, and $P^{''}_n \Rightarrow P''$, but I really don't know.]
As always, thank you for your time.
Best Answer
The direction that $P_n'\times P_n'' \Rightarrow P'\times P''$ implies the weak convergence $P_n' \Rightarrow P'$ and $P_n'' \Rightarrow P''$ is a special case of Proposition 1, since the marginals of a product measure are the respective factors. We could give a simpler proof for this special case, but I doubt that that would be very enlightening.
To show the other direction, we assume $P_n' \Rightarrow P'$ and $P_n'' \Rightarrow P''$. We use part $(i)$, which tells us that for all $P'$-continuity sets $A'$ and $P''$-continuity sets $A''$ we need to check
$$(P_n'\times P_n'')(A'\times A'') \to (P'\times P'')(A'\times A'').\tag{1}$$
But by our assumption and the Portmanteau theorem, we have $P_n'(A') \to P'(A')$ and $P_n''(A'') \to P''(A'')$, and by limit algebra it follows that
$$(P_n'\times P_n'')(A'\times A'') = P_n'(A')\cdot P_n''(A'') \to P'(A')\cdot P''(A'') = (P'\times P'')(A'\times A''),\tag{1'}$$
so indeed $(i)$ tells us that under our assumption $P_n'\times P_n'' \Rightarrow P'\times P''$.
Without using part $(i)$, we would have to show that $(P_n'\times P_n'')(A) \to (P'\times P'')(A)$ for all $(P'\times P'')$-continuity sets $A$, and there are generally a lot of $(P'\times P'')$-continuity sets that aren't products. Thus $(i)$ reduces the required work, and leaves only the case of product sets to be considered.
I think that answers 1.a. and 1.b. Concerning 1.c., in the "only if" part you should mention that you use Proposition 1, it's better to be explicit. In the "if" part, you have the right idea to use the Portmanteau theorem, but you didn't recognise how to use it.
Problem 2.7. is rather hard, since it's not clear what Billingsley expected there. I think the thing to take away is that a probability measure on a product space is in general not determined by its marginal measures. (But if all but possibly one of the marginal measures of $P$ are point masses, then $P$ is the product of its marginals.) That's in fact not hard to see once you think about it, but it's a tempting mistake.
Let's try to relate it, though. By theorem 2.8., if $P_n$ is a sequence of probability measures on the unit square converging weakly to the uniform distribution $U_{\Delta}$ on the diagonal of the unit square, then the product of the marginals, $P_n' \times P_n''$, converges weakly to the uniform distribution $U_{\square}$ on the unit square. In particular, from $P_n' \Rightarrow P'$ and $P_n'' \Rightarrow P''$ we can not conclude that $P_n \Rightarrow P$. The theorem only gives that conclusion if we additionally know that $P_n$ and $P$ are product measures, i.e. $P_n = P_n' \times P_n''$ and $P = P' \times P''$.
Ad 2.a.: We don't get counterexamples to proposition 1, since that proposition is true. What we get are counterexamples to the naïve converse of proposition 1, which would be "If $P_n' \Rightarrow P'$ and $P_n'' \Rightarrow P''$ then $P_n \Rightarrow P$". We can find counterexamples to that not only in the exact setting of problem 2.7 of course, all we need is a measure $P$ that is not a product measure, or a sequence of not-product measures whose marginals converge weakly. That can be constructed as soon as both factor spaces have more than one point.
Ad 2.b.: The weak limit - if it exists - is still unique. The point is that unlike for sequences in $\mathbb{R}^n$, the (weak) convergence of all marginals (analogous to the coordinate projections in $\mathbb{R}^n$) is no longer sufficient to deduce the (weak) convergence of the original sequence - consider $P_{2n} = U_{\Delta}$ and $P_{2n+1} = U_{\square}$ - nor, if the sequence is weakly convergent, to determine the weak limit.