You can rephrase this as a purely singular homology question by defining a product on singular homology given by the same formula you've given for cellular homology, but with singular homology. Then by naturality, if the cup product on singular homology is the same as this new product, the cup product coincides with the cellular product.
You can do this axiomatically (see, for example, Kirk and Davis's chapter on products), but you can also do it using Eilenberg-MacLane spaces.
A product on cohomology groups gives rise to a map $K(\mathbb{Z}, n) \times K(\mathbb{Z},m) \rightarrow K(\mathbb{Z},n+m)$, by Yoneda lemma. Again by Yoneda, this is classified by a class $H^{n+m}(K(\mathbb{Z}, n) \times K(\mathbb{Z},m)) \cong H^n(K(\mathbb{Z},n)) \otimes H^m(K(\mathbb{Z},m))$ because the kth Eilenberg-MacLane space is k-1 connected. By the universal coefficient theorem this is then a map $\mathbb{Z} \otimes \mathbb{Z} \rightarrow \mathbb{Z}$. This has to be given by $(a,b) \rightarrow k(ab)$.
Transferring back over to cellular cohomology (since these products are defined in the same manner), we can figure out what k is by calculating the product on $S^n \times S^m $ with its 4 cell structure. It is possible to show geometrically the Kunneth formula for cellular cohomology with ring structure given by this product, so we have that the product of the n-cell with the m-cell is the (n+m)-cell, so $k=1$.
So since these two products agree on all products of spheres, we have that the coefficients are the same in the universal case, which implies they agree for all cases.
This is an incredibly annoying subtlety. To unwind, we have to review how cup products work. Let $X$ be a topological space and $A,B\subseteq X$ be subspaces. In its most primitive form, the cup product is a map $C^i(X)\times C^j(X)\rightarrow C^{i+j}(X)$. This restricts to a cup product on relative cochains $C^i(X,A)\times C^j(X,B)\rightarrow C^{i+j}(X,A+B)$. On the other hand, there is an inclusion map $C^{i+j}(X,A\cup B)\rightarrow C^{i+j}(X,A+B)$. Now, all of these maps are compatible with the differential in an appropriate sense, hence induce maps in cohomology $H^i(X,A)\times H^j(X,B)\rightarrow H^{i+j}(X,A+B)$ and $H^{i+j}(X,A\cup B)\rightarrow H^{i+j}(X,A+B)$. The important feature now is that if $A$ and $B$ are open in $A\cup B$, then this latter map is an isomorphism. Composing with its inverse yields the true relative cup product on cohomology, $H^i(X,A)\times H^j(X,B)\rightarrow H^{i+j}(X,A\cup B)$. The important subtlety is that this inverse $H^{i+j}(X,A+B)\rightarrow H^{i+j}(X,A\cup B)$ is not given by the identity on representatives.
If we now apply this discussion to the scenario at hand, you will see that the cocycle $p_1^{\ast}\eta\cup p_2^{\ast}\eta$ represents an element of $H^2(\mathbb{R}^2,\mathbb{R}\times\mathbb{R}\setminus\{0\}+\mathbb{R}\setminus\{0\}\times\mathbb{R})$, but it does not represent an element of $H^2(\mathbb{R}^2,\mathbb{R}^2\setminus\{0\})$ directly. I leave it as an exercise to find an explicit $2$-chain (a $2$-simplex will do, actually) in $\mathbb{R}^2\setminus\{0\}$ on which $p_1^{\ast}\eta\cup p_2^{\ast}\eta$ does not vanish. Nonetheless, the cohomology class $[p_1^{\ast}\eta\cup p_2^{\ast}\eta]\in H^2(\mathbb{R}^2,\mathbb{R}\times\mathbb{R}\setminus\{0\}+\mathbb{R}\setminus\{0\}\times\mathbb{R})$ defines a cohomology class in $[\eta]\times[\eta]\in H^2(\mathbb{R}^2,\mathbb{R}^2\setminus\{0\})$, which we can pair with the generator $[\sigma]\in H_2(\mathbb{R}^2,\mathbb{R}^2\setminus\{0\})$. The downside is that we don't have an explicit representative of $[\eta]\times[\eta]$, which begs the question how we can actually concretely calculate this evaluation. Let me address this.
I will first go the long route and sketch a geometric way of understanding the inverse $H^{\bullet}(X,A+B)\rightarrow H^{\bullet}(X,A\cup B)$, which will explain how to do this calculation. Then I will give a formal argument. The chain map $C^{\bullet}(X,A\cup B)\rightarrow C^{\bullet}(X,A+B)$ is dual to the quotient map $C_{\bullet}(X,A+B)\rightarrow C_{\bullet}(X,A\cup B)$, which fits into a short exact sequence of chain complexes
$$0\rightarrow C_{\bullet}(A\cup B,A+B)\rightarrow C_{\bullet}(X,A+B)\rightarrow C_{\bullet}(X,A\cup B)\rightarrow0.$$
The complex $C_{\bullet}(A\cup B,A+B)$ is acyclic (this is the excision theorem), whence the other map induces isomorphisms in homology. Geometrically, this acyclicity comes from the fact that a cycle in $A\cup B$ possesses a subdivision, which is a cycle in $A+B$. Similarly, an inverse $H_{\bullet}(X,A\cup B)\rightarrow H_{\bullet}(X,A+B)$ can be obtained by taking a relative cycle in $(X,A\cup B)$, i.e. a chain with boundary in $A\cup B$, and subdivide it (which subdivides the boundary as well, since subdivision is a chain map) until the boundary is a chain with boundary in $A+B$, whence the subdivided cycle represents a homology class in $(X,A+B)$. Lastly, the desired inverse $H^{\bullet}(X,A+B)\rightarrow H^{\bullet}(X,A\cup B)$ is obtained by the dual of this on representatives. This is only a sketch as I'm being intentionally vague about how exactly we should subdivide each chain, etc..
Formally, the point is that the maps $H^{\bullet}(X,A\cup B)\rightarrow H^{\bullet}(X,A+B)$ and $H_{\bullet}(X,A+B)\rightarrow H_{\bullet}(X,A\cup B)$ are adjoints with respect to the pairings $H^{\bullet}(X,A+B)\times H_{\bullet}(X,A+B)\rightarrow\mathbb{Z}$ and $H^{\bullet}(X,A\cup B)\times H_{\bullet}(X,A+B)\rightarrow\mathbb{Z}$ (which is a purely algebraic fact about chain maps and their duals).
Thus, to evaluate $[\eta]\times[\eta]\in H^2(\mathbb{R}^2,\mathbb{R}^2\setminus\{0\})$ on $[\sigma]\in H_2(\mathbb{R}^2,\mathbb{R}^2\setminus\{0\})$, we choose a homology class $\alpha\in H_2(\mathbb{R}^2,\mathbb{R}\times\mathbb{R}\setminus\{0\}+\mathbb{R}\setminus\{0\}\times\mathbb{R})$ that maps to $[\sigma]$ under the canonical map (this can be achieved by appropriately subdividing a representative) and evaluate on it the cocycle $p_1^{\ast}\eta\cup p_2^{\ast}\eta$.
I leave it to you to carry this out explicitly. An appropriate subdivision of $\sigma$ will involve at least four simplices, so might be a bit fidgety. Alternatively, you could consider an appropriate subdivision of the square $[-1,1]^2$ into two simplices, which will yield an easier calculation. Of course, you can also feel free to do both calculations and confirm they yield the same result.
Best Answer
For abelian groups $G$ and $G'$, there is a map $$H^n(X;G)\otimes H^m(X;G')\to H^{n+m}(X\times X;G\otimes G')\xrightarrow{\Delta^*}H^{n+m}(X;G\otimes G')$$ for any space $X$. Recall there is a homotopy equivalence of chain complexes $f:C_*(X\times X)\simeq C_*(X)\otimes C_*(X)$ by Eilenberg-Zilber. The first map in the sequence above sends cochains $\phi:C_n(X)\to G$ and $\phi':C_m(X)\to G'$ to the cochain given by the composition $$C_{n+m}(X\times X)\xrightarrow{f_{n+m}}\bigoplus\limits_{i+j=n+m}C_i(X)\otimes C_j(X)\xrightarrow{\text{proj}}C_n(X)\otimes C_m(X)\xrightarrow{\phi\otimes\phi'}G\otimes G'$$Now suppose there is a group homomorphism $\mu:G\otimes G'\to G''$, for example the cup product $$H^i(Y;R)\otimes H^j(Y;R)\to H^{i+j}(Y;R)$$ for a space $Y$ and a ring $R$, then we get a map $$H^{n+m}(X;G\otimes G')\to H^{n+m}(X;G'')$$ via $$\bigg(\phi:C_{n+m}(X)\to G\otimes G'\bigg)\mapsto\bigg(\phi:C_{n+m}(X)\to G\otimes G'\xrightarrow{\mu}G''\bigg)$$
edit: I should remark that if we set $G=G'=G''=R$ for $R$ a ring and $\mu$ the multiplication in $R$, this is the ordinary cup product.