I'll quote the beginning of the example from the document you have linked to.
Example 3: Let $X$ be an uncountable set and let $\{0,1\}$ have the discrete topology. Consider $\mathcal P(X) = \{0,1\}^X$ with the product topology. Let $\mathcal A\subseteq \mathcal P(X)$ be the collection of all uncountable subsets of $X$. $\mathcal A$ is
not open; indeed every basic open contains finite sets. However, we claim that $\mathcal A$ is sequentially open.
The important point is notice that the author identifies $\{0,1\}^X$ with $\mathcal P(X)$. Indeed, there is a very natural bijection between these two sets (and it occurs frequently in mathematics, which might be why it is mentioned in the text without any details).
For every subset $S\subseteq X$ you have the corresponding function $\chi_S \in \{0,1\}^X$ which maps elements of $S$ to $1$ and other elements to $0$
$$\chi_S(x)=
\begin{cases}
1 & x\in S, \\
0 & \text{otherwise}.
\end{cases}
$$
The assignment $S\mapsto \chi_S$ is a bijection between $\mathcal P(X)$ and $\{0,1\}^X$.
Now if we look at the space $\{0,1\}^X$ then the basic open sets are obtained in this way: Choose finitely many coordinates $k_1,\dots,k_n$ and choose values $v_1,\dots,v_n\in\{0,1\}$ which you want to prescribe for these coordinates. Then you get a basic neighborhood consisting of all functions with these prescribed values, i.e. $f(k_j)=v_j$.
What happens if we transfer this topology (using the bijection we mentioned above) to the set $\mathcal P(X)$? Well, for every basic neighborhood we have fixed some finite set $F$ of indices where the functions have to attain the value $1$. If this function is of the form $\chi_S$, this is the same as saying $F\subseteq S$. We also prescribed some finite sets $G$ such that for $x\in G$ we want $\chi_S(x)=0$. This is the same as requiring $S\cap G=\emptyset$.
So the basic open sets in $\mathcal P(X)$ are like this: For any finite sets $F,G\subseteq X$ you get a basic set
$$\mathcal O_{F,G}=\{S\subseteq X; F\subseteq S, G\cap S=\emptyset\}.$$
You can notice, that for any choice of $F$, $G$ such that $F\cap G=\emptyset$, this basic set contains at least one finite set. (For $F\cap G\ne\emptyset$, the set $\mathcal O_{F,G}$ described here is empty.)
The product you wrote does not have to be finite because it equals the whole space $X_1\times X_2$, which is always open. An example of an open subset of $X_1 \times X_2$ would be
\begin{equation}
\left(\prod_{i\in \mathbb{Z}}A_i \right)\times X_2
\end{equation}
where $A_i$ is an open subset of $X_i$, and $A_i = X_i$ for all but finitely many $i$.
Best Answer
Given that the projection functions $p_1:X\times Y\rightarrow X$ and $p_2:X\times Y\rightarrow Y$ are continuous, take $(x,y)\in X\times Y$. Clearly: $$\{(x,y)\} = p_1^{-1}(\{x\}) \cap p_2^{-1}(\{y\})$$
But since the topologies on $X$ and $Y$ are discrete, and $p_1$ and $p_2$ are continuous, this means that $p_1^{-1}(\{x\})$ and $p_2^{-1}(\{y\})$ are open in $X\times Y$.
So the singleton set $\{(x,y)\}$ is the intersection of two open sets, and hence is an open set.
Note, this makes clear why this works for finite products, but not infinite products - a singleton in an infinite product would be the infinite intersection of open sets, which is not guaranteed to be open.