$\newcommand{\M}{M}$
$\newcommand{\N}{N}$
$\newcommand{\brk}[1]{\left(#1\right)}$
$\newcommand{\be}{\beta}$
$\newcommand{\al}{\alpha}$
$\newcommand{\til}{\tilde}$
The pullback bundle is indeed an embedded submanifold of the product $M \times E$. The essential notion here is transversality. (Together with the fact that bundle projection is a submersion).
The full story with all the details is a bit long; I haven't seen it done in any textbook, I found all the steps here and there, and built my own picture of the things.
Definitions
$(1)$ Let $\M,\N$ be a smooth manifolds. Suppose $F:\N \to \M$ is a smooth map, $S \subseteq \M$ is an embedded submanifold. We say $F$ is transverse to $S$ if $\forall x \in F^{-1}\brk{S} \, , \, T_{F\brk{x}}\M= T_{F(x)}S + dF_x(T_x\N)$.
$(2)$ Let $\M,\N,\N'$ be smooth manifolds. Suppose $F:\N \to \M \, , \, F':\N' \to \M$ are smooth maps. We say $F,F'$ are transverse to each other if $\forall x \in \N, x' \in \N' $ such that $F(x)=F'(x')$ , $T_{F(x)}\M=dF_x(T_x\N) + dF'_{x'}(T_{x'}\N')$.
Note: If either one of $F,F'$ is a submersion, then they are automatically transverse.
Note: Some of ther proofs are at the end of the answer (So it will be possible to skim through the general scheme without all the details at first)
Lemma (1):
$\M,\N$ be a smooth manifolds, $S \subseteq M$ is an embedded submanifold. Let $F:\N \to \M$ be transverse to $S$. Then $F^{-1}(S)$ is an embedded submanifold of $\N$ whose codimension is equal to the codimension of $S$ in $\M$.
proof:
See Theorem 6.30 , in Lee. (pg 144).
Lemma (2): (This is exercise 13 in chapter 6, Lee)
Let $\M,\N,\N'$ be smooth manifolds. Suppose $F:\N \to \M \, , \, F':\N' \to \M$ are smooth maps. Then $F,F'$ are transverse to each other if and only if the map $F \times F' : \N \times \N' \to \M \times \M$ is transverse to the diagonal $\Delta_\M = \{(x,x)|x \in \M \}$
Lemma (3):
Let $\M$ be a smooth manifold. Then $\Delta_\M = \{(x,x)|x \in \M \}$ is an embedded (smooth) submanifold of $\M \times \M$.
Lemma (4):
Let $\M$ be a manifold. Let $\Delta_\M$ be the diagonal manifold of $\M$. (see Lemma 3 ). Then $T_{\brk{x,x}}\Delta_\M = \text{diag}\brk{T_x\M \times T_x\M}=\{(v,v)| v \in T_xM \}$. (i.e the tangent space of the diagonal is the diagonal of the tangent space).
proof:
Since any tangent vector can be realized as a derivative of a path, the tangent space to a manifold is identical to the set of derivatives of paths. Since $\Delta_\M$ is an embedded submanifold, a path $\be:I \to \Delta_\M $ is smooth if and only if it is smooth when considered as a path into the product $\M \times \M$ if and only if each of its components is smooth. So $\be(t)=\brk{\al\brk{t},\al\brk{t}}$ , where $\al : I \to \M$, so $\dot \be (0) \overset{(*)}= \brk{\dot \al (0),\dot \al (0)}$ , hence its clear the tangent space to the diagonal is exactly the diagonal of the tangent space. (Where in (*) we used the canonical isomorphism between $T_{(x,x')}\brk{\M \times \M'} = T_x\M \oplus T_{x'}\M'$ via the differentials of the projections onto the different components).
corollary (1):
Let $\M,\N,\N'$ be smooth manifolds, $F:\N \to \M \, , \, F':\N' \to \M$ are smooth maps. (In short we write $\N\overset{F}{\rightarrow}\M\overset{F'}{\leftarrow}\N'$). Assume $F,F'$ are transverse to each other. Then the fiber product of this diagram, which is defined as $\{\brk{x,x'} \in \brk{\N, \N'}|F(x)=F'(x') \}$ is an embedded smooth submanifold of the product $\N \times \N'$.
proof of corollary (1):
The fibered product $\N \times_{\M} \N'$ is the inverse image $(F \times F')^{-1}\brk{\Delta_{\M}}$. By Lemma 3, $\Delta_\M$ is a submanifold of $\M \times \M$. Now combine Lemma 2 and Lemma 1.
corollary (2):
Let $\M,\N,\N'$ be smooth manifolds, $\N\overset{F}{\rightarrow}\M\overset{F'}{\leftarrow}\N'$. If either one of $F,F'$ is a submersion, then the fiber product $\N \times_\M \N' = \{\brk{x,x'} \in \brk{\N, \N}|F(x)=F'(x') \}$ is an embedded submanifold of the product $\N \times \N'$.
proof of corollary (2):
If one of $F,F'$ is a submersion, then these two maps are automatically transverse to each other. Now use corollary (1).
In particular we get the following proposition:
Let $\pi: E \to B$ be a vector bundle, $f:B' \to B$. The pullack bundle $f^*\brk{E}$ is an embedded submanifold of the product $B' \times E$.
(This is becuse the bundle projection $\pi$ is always a submersion).
proof of Lemma (2):
First, we need a sublemma:
Sub-lemma:
Let $V$ be a vector space, $V_1,V_2 \subseteq V$ are subspaces. Let $\text{diag}(V \times V) = \{(v,v)|v \in V \} $. Then
$V \oplus V = \text{diag}(V \times V) + \brk{V_1 \oplus V_2} \iff V = V_1 + V_2$
proof of the sublemma:
$\Rightarrow :$ Let $v \in V$. Then $(v,0) \in V \oplus V$, hence by our assumption $\exists \til v \in V, v_1 \in V_1 , v_2 \in V_2$ such that $(v,0) = (\til v ,\til v) + (v_1,v_2)=(\til v + v_1, \til v + v_2) \Rightarrow \til v = -v_2, v = \til v + v_1 = v_1 - v_2 \in V_1 + V_2 $ .
$\Leftarrow :$ Note that both sides of the left equation are subspaces. Hence, from symmetry it's enough to show that $\forall v \in V \, , \, (v,0) \in \text{diag}(V \times V) + \brk{V_1 \oplus V_2}$. The assumption $V =V_1 + V_2 \Rightarrow \exists v_i \in V_i$ such that $v = v_1 -v_2$. Define $\til v = -v_2$, so we get $(v,0)=(v_1-v_2,\til v +v_2)=(v_1 + \til v, v_2 + \til v) = (\til v,\til v) +(v_1,v_2)$.
Now to the actual proof of Lemma (2):
By definition (1), $F \times F'$ is transverse to the diagonal if
\begin{split}
&\forall (x,x') \in (F \times F')^{-1}\brk{\Delta_\M} \, , \, T_{(F \times F')\brk{x,x'}}\brk{\M \times \M}= T_{(F \times F')(x,x')}\Delta_\M + d(F \times F')_{(x,x')}(T_{(x,x')}\brk{\N \times \N'}) \iff \\
& T_{\brk{F(x),F'(x')}}\brk{\M \times \M}= T_{\brk{(F(x),F'(x')}}\Delta_\M + d(F \times F')_{(x,x')}(T_x\N \oplus T_x\N') \iff \\
& T_{\brk{F(x),F(x)}}\brk{\M \times \M}= T_{\brk{(F(x),F(x)}}\Delta_\M + d(F \times F')_{(x,x')}(T_x\N \oplus T_x\N') \iff \\
&T_{\brk{F\brk{x}}}\M \oplus T_{\brk{F\brk{x}}}\M \overset{Lemma 4}= \text{diag}\brk{T_{F(x)} \M \times T_{F(x)}\M} + \brk{ dF_x \brk{T_x \N} \oplus dF'_{x'} \brk{T_{x'}\N'}} \overset{Sub-lemma} \iff \\
&T_{F(x)}\M =dF_x \brk{T_x \N} + dF'_{x'} \brk{T_{x'}\N'}
\end{split}
Since the last row is the defintion transverse maps, we finished.
proof of Lemma (3):
The diagonal is the graph of the smooth function $Id_\M$, and graphs of smooth functions are always embedded submanifolds of the product of the domain and the codomain. (See prop 5.4, Lee).
Best Answer
I'd like to thank Jordan Payette for the crucial help.
Locally, $R_\phi$ is represented by points $(x,(d\phi)^*\eta,\phi(x),\eta)$, hence it is parametrized by coordinates $x,\eta$. This proves that $R_\phi$ is embedded in $T^*Q_1\times T^*Q_2$ and has dimension $n_1+n_2$ (i.e., half the dimension of the ambient space). So in order to prove that $R_\phi$ is Lagrangian, it suffices to show that it is isotropic, i.e., $i^*\omega=0$, where $i:R_\phi\hookrightarrow T^*Q_1\times T^*Q_2$ with $ (x,\eta)\mapsto (x,(d\phi)^*\eta,\phi(x),\eta)$ is the inclusion.
If $u(x,\eta):=(d\phi)_x^*(\eta)$, notice that: \begin{align*} u_i&=u(x,\eta)\left(\frac{\partial}{\partial x_i}\right)=\eta\left(d\phi\left(\frac{\partial}{\partial x_i}\right)\right)=\sum_{j=1}^{n_2}\eta_j\frac{\partial \phi_j}{\partial x_i}\\ \frac{\partial u_i}{\partial x_k}-\frac{\partial u_k}{\partial x_i}&=\sum_{j=1}^{n_2}\eta_j\frac{\partial^2 \phi_j}{\partial x_k\partial x_i}-\eta_j\frac{\partial^2 \phi_j}{\partial x_i\partial x_k}=0\\ \frac{\partial u_i}{\partial \eta_k}&=\frac{\partial \phi_k}{\partial x_i} \end{align*}
Hence: \begin{align*} i^*\omega &=i^*\left(\sum_{i=1}^{n_1}dx_i\wedge d\xi_i-\sum_{j=1}^{n_2}dy_j\wedge d\eta_j\right)\\ &=\sum_{i=1}^{n_1}dx_i\wedge du_i-\sum_{j=1}^{n_2}d\phi_j\wedge d\eta_j\\ &=\sum_{i=1}^{n_1}dx_i\wedge\left(\sum_{k=1}^{n_1}\frac{\partial u_i}{\partial x_k}dx_k+\sum_{j=1}^{n_2}\frac{\partial u_i}{\partial\eta_j}d\eta_j\right)-\sum_{j=1}^{n_2}\left(\sum_{i=1}^{n_1}\frac{\partial\phi_j}{\partial x_i}dx_i\right)\wedge d\eta_j\\ &=\sum_{i=1}^{n_1}\sum_{k=1}^{n_1}\frac{\partial u_i}{\partial x_k}dx_i\wedge dx_k+\sum_{i=1}^{n_1}\sum_{j=1}^{n_2}\underbrace{\left(\frac{\partial u_i}{\partial\eta_j}-\frac{\partial\phi_j}{\partial x_i}\right)}_{=0}dx_i\wedge d\eta_j\\ &=\sum_{1\leq i<k\leq n_1}\underbrace{\left(\frac{\partial u_i}{\partial x_k}-\frac{\partial u_k}{\partial x_i}\right)}_{=0}dx_i\wedge dx_k=0 \end{align*}
We prove that $R_\phi$ is diffeomorphic to $N^*\Gamma_\phi$, where $\Gamma_\phi:=\text{graph}(\phi)$. If $i':\Gamma_\phi\hookrightarrow Q_1\times Q_2$ with $x\mapsto (x,\phi(x))$ is the inclusion, we have $di'\left(\frac{\partial}{\partial x_i}\right)=\frac{\partial}{\partial x_i}+d\phi\left(\frac{\partial}{\partial x_i}\right)$, which means $T_{(x,\phi(x))}\Gamma_\phi=\text{span}\left(\frac{\partial}{\partial x_i}+d\phi\left(\frac{\partial}{\partial x_i}\right)\right)$. Moreover, $T^*(Q_1\times Q_2)\simeq T^*Q_1\times T^*Q_2$ and $N^*\Gamma_\phi=\{(x,\xi)\in T^*(Q_1\times Q_2)\mid x\in \Gamma_\phi,\,\xi|_{T_x\Gamma_\phi}\equiv 0\}$. Therefore:
\begin{align*} N^*\Gamma_\phi &\simeq\{(x,\xi,y,\eta)\mid (x,y)\in \Gamma_\phi,\,(\xi,\eta)|_{T_{(x,y)}\Gamma_\phi}\equiv 0\}\\ &=\left\{(x,\xi,y,\eta)\mid y=\phi(x),\,\xi\left(\frac{\partial}{\partial x_i}\right)+\eta\left(d\phi\left(\frac{\partial}{\partial x_i}\right)\right)=0\,\forall i\right\}\\ &=\left\{(x,\xi,y,\eta)\mid y=\phi(x),\,(\xi+(d\phi)^*\eta)\left(\frac{\partial}{\partial x_i}\right)=0\,\forall i\right\}\\ &=\{(x,\xi,y,\eta)\mid y=\phi(x),\,\xi=-(d\phi)^*\eta\}\subset T^*Q_1\times T^*Q_2 \end{align*} Since $f:(x,\xi,y,\eta)\mapsto (x,\xi,y,-\eta)$ is a diffeomorphism (in fact a symplectomorphism) from $(T^*Q_1\times T^*Q_2,\omega_1\oplus\omega_2)$ to $(T^*Q_1\times T^*Q_2,\omega_1\oplus-\omega_2)$, we get a diffeomorphism: $$\left.f\right|_{N^*\Gamma_\phi}:N^*\Gamma_\phi\to R_\phi$$