Such triples, involving two objects -or even categories- and some kind of an "action" of one of them on the other (respecting some or all of its structure maps), are quite general and are met -as has already been indicated in the other answers and the comments as well- in various different branches of mathematics and mathematical physics. A fair coverage might deserve a suitably sized expository article.
I will try to sketch some of these ideas, and the corresponding objects, but it is important to note that there are lots of others as well (just to mention a few: $C^*$-algebras, graded algebras, etc, provide somewhat similar examples in different directions).
$\blacksquare$ Case $I$: The smash product $A\sharp H$, between a hopf algebra $H$ and a $H$-module algebra $A$
One notion which is similar and in some sense generalises the first construction discussed in the OP, is the smash product algebra, between a Hopf algebra $H$ and an algebra $A$, which also a $H$-module algebra:
Definition: Let $Η$ be a $k$-Hopf algebra and $Α$ a $k$-algebra,
which is also a $H$-module algebra. The smash product of $Α$ and $Η$, denoted by
$Α \sharp Η$ will be the algebraic structure determined by the following data:
1. As $k$-vector spaces $Α \sharp Η = Α \otimes_{k} Η$. We will denote the element $a \otimes h$ as $a \sharp h$.
2. For any $a, b \in A$ and for any $h, g \in H$ define a bilinear multiplication by:
$$
(a \sharp h)(b \sharp g) =
\sum a(h_{(1)} \cdot b) \sharp h_{(2)}g
$$
where $\Delta(h)=\sum h_{(1)}\otimes h_{(2)}$ the comultiplication in $H$.
It is relatively easy to show that the $k$-v.s. $Α \sharp Η$ equipped with the above defined multiplication becomes an associative $k$-algebra with unity. This algebra is called the smash product algebra or crossed product algebra between the hopf algebra $H$ and the $H$-module algebra $A$.
It can furtherore be shown that -under suitable circumstances- $Α \sharp Η$ also becomes a $k$-Hopf algebra called the crossed product hopf algebra.
(in the above $k$ is the field).
Now what might be particularly interesting, is that the above smash product algebra, generalizes some similar but well known constructions. So let us study closer some specific cases of smash product algebras:
- Example 1: The skew group algebra
Let $Α$ be a (left) $kG$-module algebra. Using the definition above, we get that for all $g \in G$ we have:
$$
g\cdot (ab) = (g \cdot a)(g \cdot b)
$$
i.e.: the elements of $G$ act as automorphisms of $Α$. We thus get a group homomorphism:
$$
\sigma: G \longrightarrow Aut_{k}A
$$
and conversely: given any automorphism $\sigma: G \rightarrow Aut_{k}A$, $Α$
becomes a (left) $kG$-module algebra. By the definition of the multiplication in $A \sharp kG$ :
$$
(a \sharp g)(b \sharp h) = a(g \cdot b) \sharp gh = a \sigma(g)(b)
\sharp gh
$$
which proves that, in this case the smash product algebra is isomorphic to the skew group algebra:
$$A \sharp kG \cong A \star G$$
as $k$-algebras.
The next example, may be seen as a special case of the above:
- Example 2: The semidirect product of groups
Let $Κ$ and $Η$ two groups and let us assume that the elements of $K$ act as automorphisms of $H$. (We thus have a group homomorphism: $\phi: Κ \rightarrow Aut(H)$). Thus, the elements of $Κ$ act as automorphisms of the group algebra $kH$, which becomes a $kK$-module algebra. From the definition of the multiplication in $kH \sharp kK$ we get that for all $h, h' \in H$ και $g, g' \in K$:
$$
(h \sharp g)(h' \sharp g') = h(g \cdot h') \sharp gg' = h \phi(g)(h') \sharp gg'
$$
i.e., in this case:
$$
kH \star K \cong kH \sharp kK \cong
k(H \rtimes_{\phi} K)
$$
where $H \rtimes_{\phi} K$ stands for the semidirect product of the groups $Η$, $Κ$ (which is a group itself) and $k(H \rtimes_{\phi} K)$ is its group algebra.
A remarkable consequence, is that, if the action
$\phi$ is trivial, in the sense that, for all $g \in K$ and for all $h
\in H$:
$$
g \cdot h = \phi(g)(h) = h
$$
then the semidirect product is identified with the direct product (as groups) and the smash product is identified with the tensor product (as algebras). In that case, the former relation reproduces the well known (from group representation theory):
$$
kH \otimes kK \cong k(H \times K) \equiv k(H \oplus K)
$$
On the other hand, the smash product algebra described above, plays a much more important role in the structure theory of hopf algebras than simply generalizing or mimicking other known similar constructions: For example, it is well known that any cocommutative hopf algebra $H$ over an algebraically closed field of characteristic zero is isomorphic to the smash product hopf algebra between the group hopf algebra of its grouplikes $G(H)$ and the UEA of the Lie algebra of its primitives $U(P(H))$:
$$
H\cong U(P(H))\sharp kG(H)
$$
This is commonly refered to as the Cartier-Kostant-Milnor-Moore theorem.
---------------------------------------------------------------------------------------------------------
$\blacksquare$ Case $II$: The crossed product $A\sharp_\sigma kG$, between an algebra $A$ and a finite group $G$
Another similar construction of crossed products, not -in principle- related with the Hopf algebra case and its module algebra described in the former case, is the following:
Let $A$ be a finite dimensional algebra over an algebraically closed field and $G$ a finite group.
Let also, the following additional data:
$\bullet$ a convolution-invertible map $\sigma:G\times G\rightarrow A$ (Note that: $\sigma\in Hom_k(kG\otimes kG,A)$)
$\bullet$ a $k$ linear map $kG\otimes A\rightarrow A$, denoted $g\otimes a\mapsto g\cdot a$
satisfying:
$$
g\cdot(h\cdot a)= \sigma(g,h)(gh\cdot a)\sigma^{-1}(g,h) \rightsquigarrow twisted \ module \ condition \\
\big(g\cdot\sigma(h,l)\big)\sigma(g,hl)=\sigma(g,h)\sigma(gh,l) \rightsquigarrow cocycle \ condition \\
g\cdot(ab)=(g\cdot a)(g\cdot b) \rightsquigarrow kG \ measures \ A \\
\sigma(1,g)=\sigma(g,1)=1, \ \ g\cdot1=1, \ \ 1\cdot a=a
$$
for all $g,h,l\in G$ and for all $a,b\in A$. Thus, $\sigma$ is a $2$-cocycle.
If we denote the element $a\otimes g\in A\otimes kG$ by $a\sharp g$, then we can define a product on this vector space by:
$$
(a\sharp g)(b\sharp h)=a(g\cdot b)\sigma(g,h)\sharp gh
$$
This definition results in an associative algebra called crossed product algebra and denoted by $A\sharp_\sigma kG$.
It is a fully $G$-graded algebra, with $g$-component: $(A\sharp_\sigma kG)_g=A\sharp_\sigma kg$ for all $g\in G$ and with the algebra $A$ embedding in it as the identity component. You can see some discussion about it in the article Irreducible representations of crossed products, J. of Pure and applied algebra, v.129, 3, p. 315-326, 1998, by S. Montgomery and S.J. Witherspoon.
---------------------------------------------------------------------------------------------------------
$\blacksquare$ Generalization of Cases $I$ and $II$: The crossed product $A\sharp_\sigma H$, between an algebra $A$ and a hopf algebra $H$, which "measures" $A$
We will say that a hopf algebra $H$ measures the algebra $A$, if there is a $k$-linear map $H\otimes A\to A$, given by $h\otimes a \mapsto h\cdot a$ and such that $h\cdot 1=\epsilon(h)1$ and $h\cdot (ab)=\Sigma(h_{(1)}\cdot a)(h_{(2)}\cdot b)$ for all $h\in H$, $a,b\in A$.
Definition: Assume a triple $(A,H, \sigma)$ consisting of the following data:
- An algebra $A$,
- A hopf algebra $H$ measuring $A$ and
- A convolution invertible map $\sigma\in Hom_k(H\otimes H, A)$
The crossed product $A\sharp_\sigma H$, of $A$ with $H$, is the vector space $A\otimes H$ equipped with the product
$$
(a\sharp h)(b\sharp k)=\sum a(h_{(1)} \cdot b)\sigma(h_{(2)},k_{(1)}) \sharp h_{(3)}k_{(2)}
$$
for all $a,b\in A$ and for all $h,k\in H$.
Note that now, unlike case $I$, we do not demand $A$ to be an $H$-module. Based on the above definition, the next lemma can be shown in a relatively straightforward manner:
Lemma: The crossed product $A\sharp_\sigma H$, of $A$ with $H$, is an associative algebra with identity $1\sharp 1$ if and only if the following two conditions are met:
- $A$ is a twisted $H$-module, that is $1\cdot a=a$, for all $a\in A$ and
$$
h\cdot(k\cdot a)= \sum \sigma(h_{(1)},k_{(1)})(h_{(2)}k_{(2)}\cdot a)\sigma^{-1}(h_{(3)},k_{(3)})
$$
for all $h,k\in H$, $a\in A$.
- $\sigma$ is a cocycle, that is, $\sigma(1,h)=\sigma(h,1)=\epsilon(h)1$ and
$$
\sum \big( h_{(1)}\cdot\sigma(k_{(1)},m_{(1)}) \big)\sigma(h_{(2)},k_{(2)}m_{(2)})=\sum\sigma(h_{(1)},k_{(1)})\sigma(h_{(2)}k_{(2)},m)
$$
for all $h,k,m\in H$.
In case $\sigma$ is trivial, in the sense that $\sigma(h,k)=\epsilon(h)\epsilon(k)1$ for all $h,k\in H$, then the above conditions directly imply that $A$ is an $H$-module algebra and that the crossed product becomes the smash product
$$
A\sharp_\sigma H=A\sharp H
$$
of case $I$, discussed above.
In case $H=kG$, where $G$ is a finite group, then the crossed product becomes the group crossed product
$$
A\sharp_\sigma H=A\sharp_\sigma kG
$$
of case $II$, discussed above.
(More details on the properties of the structure and representations of the crossed product algebra $A\sharp_\sigma H$ can be found in ch. 7 of S. Montgomery's book "Hopf algebras and their actions on rings").
Best Answer
Suppose you define some equivalence relation on $X$ so that the quotient $\overline{X}$ has the property that $N\rtimes Q$ acts on $\overline{X}\times Y$ coordinatewise, with the action of $n$ on $[x]$ being $[nx]$ (here, $[x]$ is the equivalence class of $x$).
Writing ${}^qn$ for $\varphi(q)(n)$, and writing the elements of $N\rtimes Q$ as pairs $(n,q)$, we would need $[nx] = [{}^qnx]$ for all $q\in Q$ (with fixed $x$ and $n$). For $({}^qn,1) = (1,q)(n,1)(1,q^{-1})$, and looking at the actions on a pair $([x],y)$ readily gives the need for $[nx]=[{}^qnx]$.
Conversely, if $\sim$ is any equivalence relation on $x$ that satisfies this property, then the action on $\overline{X}\times Y$ given by $(n,q)([x],y) = ([nx],y)$ is readily seen to be an action: $$\begin{align*} (n_1,q_1)\Bigl((n_2,q_2)([x],y)\Bigr) &= (n_1,q_1)([n_2x],q_2y)\\\ &= ([n_1(n_2x)],q_1(q_2y))\\\ \Bigl((n_1,q_1)(n_2,q_2)\Bigr)([x],y) &= (n_1{}^{q_1}n_2,q_1q_2)([x],y)\\\ &= ([n_1{}^{q_1}n_2 x],(q_1q_2)y). \end{align*}$$ The assumption on $\sim$ gives that $[{}^{q_1}n_2 x] = [n_2x]$, and hence that $[n_1{}^{q_1}n_2 x] = [n_1n_2x]$.
If you let $\sim$ be the transitive closure of the relation with $x\sim x'$ if and only if there exists $x_0\in X$, $n\in N$, and $q\in Q$ with $nx_0 = x$, ${}^qnx_0 =x'$, then any equivalence relation that is coarser than $\sim$ will do. (I may have my "coarser" and "finer" mixed up here; I mean that the equivalence classes are unions of equivalence classes under $\sim$, so that if you think of it as if it were a "topology", you would have a coarser topology, with fewer open sets).
In particular, taking $\sim$ to be the total relation ($x\sim x'$ for all $x,x'\in X$), you get the natural action on $Y$ given by mapping to the quotient (well, technically an action on $\{\bullet\}\times Y$, but that is naturally isomorphic to an action on $Y$). If the action of $Q$ on $N$ is trivial, so that ${}^qn = n$, then $\sim$ is the identity relation, so $\overline{X}=X$ and you get the natural action on $X\times Y$.