The enlargement axiom does not trivialise the theory. It does imply that $X$ is a neighbourhood of every $x \in X$: every $x \in X$ has some $N \in N(x)$ (this is actually part of the axioms/conditions, often forgotten: every $N(x)$ is non-empty) and $N \subseteq X$ so $X \in N(x)$ by axiom 2. And everything inbetween $N$ and $X$ is a neighbourhood too. We cannot go smaller in general, so it's not true that everything becomes a neighbourhood of everything: $X$ is a neighbourhood of every point, but any fixed set $A$ is only a neighbourhood of $x$ if it happens to be the case that $A \in N(x)$, and otherwise it's not.
As to the necessity of the axiom: look at metric spaces, where $A$ is a neighbourhood of $x$ iff there is some $r > 0$ such that $B(x,r) \subseteq A$. The way this is defined already implies that larger sets than $A$ are also neighbourhoods of $x$; the same $r$ will work. So it's not a very surprising axiom. The same can be said for ordered spaces ($A$ is a neighbourhood of $x$iff either there exists $a \in X$ with $x < a$ and $\{z : z < a\} \subseteq A$, or there exists $b \in X$ with $b < x$ and $\{z : b < z \} \subseteq A$ or there exist $a,b \in X$ with $a < x, x < b$ and $\{z: a < z, z < b \} \subseteq A$), where also the enlargement is clearly true. We really are interested in the "small" neighbourhoods of $x$ (for continuity etc.) but in this axiomatisation we want to capture the notion of all possible neighbourhoods, even the not so interesting large ones. We could also axiomatise "basic" neighbourhoods instead, instead of all neighbourhoods, but this is not what we are doing here. In short, the relation of $A$ being a neighbourhood of $x$ is "local" and not affected by enlarging the set.
As to axiom 4, it is true that the "intended" result is that every neighbourhood of $x$ contains an open neighbourhood of $x$, but for that we first need to define what "open" means. We only have a set of sets $\{N(x): x \in X\}$ satisfying the axioms, no notion of openness. But if you already know the standard axioms for topology in terms of open sets, you want to go back and forth:
If you have a topology $\mathcal{T}$, define for each $x \in X$: $N(x) = \{A \subseteq X: \exists O \in \mathcal{T}: x \in O \subseteq A\}$ and check that this fulfills the axioms (again see that 2 is trivially fulfilled). Call this system $\mathcal{N}(\mathcal{T})$. On the other hand, if we have a system $\mathcal{N}= \{N(x): x \in X \}$ satisfying the axioms, define $\mathcal{T} = \{O \subseteq X: \forall x \in O: O \in N(x) \}$ and check that this defines a topology (we do not need axiom 4 for this, it turns out). Call this topology $\mathcal{T}(\mathcal{N})$. The axioms 4 is used to show that $\mathcal{N}(\mathcal{T}(\mathcal{N})) = \mathcal{N}$ again. So the neighbourhood system from the topology induced by the neighbourhood system is that same one we started with. It's a fun exercise to check this out in detail. It will enhance your understanding of these axioms. It's also true that $\mathcal{T}(\mathcal{N}(\mathcal{T})) = \mathcal{T}$, which is also fun to prove.
You’re right that $D[x]$ need not be an open neighbourhood of $x$. As a simple example of this, note that $D=\{(u,v) \in \mathbb{R}^2: |u-v| \le \varepsilon\}$ is an entourage for $\mathbb{R}$ in the standard uniformity induced by the metric $d(x,y)=|x-y|$. And $D[x]= \{y\in \mathbb{R}: |x-y| \le \varepsilon\} = [x-\varepsilon, x+\varepsilon]$ is not an open neighbourhood of $x$. This is basically due to the fact that entourages are closed upwards for inclusion, the basic entourages induced by a metric are usally set to be of the form $D=\{(x,y): d(x,y) < r\}$ and these induce just the usual (open) balls as neighbourhoods.
As to the product $X \times X$, where we see $X$ just as a topological space (as James also does). This has a base of open sets of the form $U \times V$ where $U,V$ are open subsets of $X$, as is well-known. This implies that a set of the form $N_x \times N_y$ is a neighbourhood of $(x,y)$ in $X \times X$ iff $N_x$ is a neighbourhood of $x$ and $N_y$ is a neighbourhood of $y$:
Proof If $N_x \times N_y$ is of that form, then there is an open set $O_x$ of $X$ such that $x \in O_x \subseteq N_x$ and an open set $O_y$ of $X$ such that $y \in O_y \subseteq N_y$. This is just the definition of $N_x$ resp. $N_y$ being a neighbourhood of $x$ resp. $y$ in a topological space. But then of course $O_x \times O_y$ is product open in $X \times X$ and $(x,y) \in O_x \times O_y \subseteq N_x \times N_y$, so again by the definition of neighbourhood we have that $N_x \times N_y$ is a neighbourhood of $(x,y)$ in $X \times X$. On the other hand, if we know $N_x \times N_y$ is a neighbourhood of $(X,y)$ in $X \times X$ there must be a product open set $O$ such that $(x,y) \in O \subseteq N_x \times N_y$. So by the definition of a base there exists a basic open set $U \times V$ in $X \times X$ (so $U$ and $V$ are open sets of $X$) such that $$(x,y) \in U \times V \subseteq O \subseteq N_x \times N_y$$ It follows easily that then $x \in U \subseteq N_x$ and $y \in V \subseteq N_y$, showing that indeed $N_x$ and $N_y$ are neighbourhoods of $x$ resp. $y$ and we are done.
Another easy observation which holds for all topological spaces: if $A \subseteq X$ and for each $x \in A$ we have a neighbourhood $N_x$ of $x$ in $X$, then $\bigcup_x N_x$ is a neighbourhood of $A$ in $X$. This is immediate from the definition of neighbourhoods and the fact that open sets are closed under all unions.
Now, what James has shown in the quoted computation is in fact that if $D$ is an entourage and $M \subseteq X \times X$:
$$D \circ M \circ D = \bigcup\{D[\xi] \times D[[\eta]: (\xi, \eta) \in M \}$$
which by all the previous remarks is just a union of product neighbourhoods of all points of $M$ and so a neighbourhood of $M$ in the product.
Hope this clarifies the matter somewhat. It’s mostly observations on neighbourhoods in general that are needed here, plus knowing what basic open sets in the product are.
Best Answer
Not necessary even when $U=\Bbb R$. Indeed, consider a neighborhood $$X=\{(x, y,z,t)\in \Bbb C^2: (1+|x|)(1+|z|)(|y|+|t|)<1\}$$ of the set $U\times U$. Suppose to the contrary that there exists a neighborhood $V$ of $U$ such that $V\times V\subset X$. Pick an arbitrary point $(x,y)\in V$ with $|y|>0$. Then a point $(1/y, 0)\in \Bbb R\subset V$, so $(x,y,1/y,0)\in V\times V\subset X$. But $$(1+|x|)(1+1/|y|)|y|\ge 1+|y|>1,$$ a contradiction.