Your argument does not prove the result at this point. For a bounded linear operator, $T$ is self-adjoint ($T^*=T$) iff $(Tx,y)=(x,Ty)$ for every $x,y\in H$. So you need to do more.
Using the textbook lemma is a good idea. It yields a straightforward argument. The proof of that lemma is not very difficult if you are familiar with polarization. It boils down to observing that
$$
2(Qx,y)=(Q(x+y),x+y)-i(Q(x+iy),x+iy)=0\qquad\forall x,y\in H.
$$
Then for $y=Qx$, you get $(Qx,Qx)=\|Qx\|^2=0$, whence $Qx=0$ for every $x$, i.e. $Q=0$. So you just have to observe that $Q=T-T^*$ satisfies the condition $$(Qx,x)=(Tx,x)-(T^*x,x)=(Tx,x)-(x,Tx)=(Tx,x)-\overline{(x,Tx)}=(Tx,x)-(Tx,x)=0.$$
The lemma proves that $Q=T-T^*=0$, i.e. $T=T^*$ as desired.
You could also use polarization-like formulae to prove directly that $(Tx,y)=(x,Ty)$. But I think using this lemma is better.
Note that the lemma is false in the real case. Think of $\pi/2$ rotations. For instance, on $\mathbb{R}^2$:
$$
Q=\pmatrix{0&-1\\1&0}\quad\Rightarrow\quad (Qx,x)=-x_2x_1+x_1x_2=0\quad\forall x\in\mathbb{R}^2.
$$
Adjoints of closed densely-defined linear operators on a Hilbert space $X$ are nice, once you get used to working in the graph space. In fact, the proofs are easier using these techniques for general closed densely-defined operators than the special-case proofs offered for the bounded case. John von Neumann introduced this way of working with densely-defined linear operators on a Hilbert space $X$. I'll explain his approach.
A Graph: The first thing to observe is that a subspace $\mathcal{M}\subseteq X\times X$ is the graph $\mathcal{G}(L)=\{ \langle x, Lx\rangle : x\in\mathcal{D}(L)\}$ of a linear operator $L : \mathcal{D}(L)\subseteq X\rightarrow X$ iff $\langle 0,y\rangle \in \mathcal{M}$ implies $y=0$. And $L$ is a closed linear operator iff its graph is a closed in the product space $X\times X$. If the subspace $\mathcal{M}$ is a graph, then the domain of $L$ becomes the set of all first coordinates in the subspace $\mathcal{M}$. It is easy to show that this unique correspondence between $x \in \mathcal{D}(L)$ and the second coordinate is linear because $\mathcal{M}$ is linear.
Closable: A linear operator $L : \mathcal{D}(L)\subseteq X\rightarrow X$ is closable iff the closure $\mathcal{G}(L)^{c}$ in $X\times X$ of its graph $\mathcal{G}(L)$ is a graph. Equivalently, if $\{ x_{n} \}\subseteq \mathcal{D}(L)$ converges to $0$ and $\{ Ax_{n} \}$ converges to some $y$, then $y=0$. That's the condition for a linear operator to have a closed extension, and is equivalent to the requirement that the closure of the graph of $L$ be the graph of a linear operator.
Inverses: If $L$ is a linear operator, then $L^{-1}$ exists iff the transpose of the graph of $L$ is a graph. Using the transpose map $\tau\langle x,y\rangle = \langle y,x\rangle$,
$$
\tau \mathcal{G}(L)=\mathcal{G}(L^{-1}),
$$
provided both exist. Notice that if $L^{-1}$ exists, then $L^{-1}$ is closed iff $L$ is closed because $\tau$ is a unitary map on $X\times X$ with $\tau^{2}=I$.
Adjoints: The real power of using graphs comes in defining the adjoint. If $L$ is a closed densely-defined linear operator, then $L$ has a closed densely-defined adjoint $L^{\star}$ whose graph is
$$
\mathcal{G}(L^{\star})=J[\mathcal{G}(L)^{\perp}],
$$
where the orthogonal complement is taken in $X\times X$ and $J$ is the unitary symplectic transpose
$$
J\langle x, y\rangle = \langle y, -x\rangle.
$$
Notice that $J^{2}=-I$. Because $\tau$ and $J$ are unitary, they commute with the action of taking orthogonal complement. So you may also write
$$
\mathcal{G}(L^{\star})=J[\mathcal{G}(L)^{\perp}]=[J\mathcal{G}(L)]^{\perp}.
$$
Of course the transpose operator $\tau$ also commutes with the action of taking the orthogonal complement.
Commutativity: Also note that $\tau J = - J\tau$, which means $\tau J\mathcal{M}=J\tau\mathcal{M}$ for subspaces $\mathcal{M}$ of $X\times X$. So, when you are considering the action of $J$, $\tau$ and $\perp$ on subspaces $\mathcal{M}\subseteq X\times X$, you can freely interchange these operations. That makes life very simple.
Your Example: Suppose that $L$ is a closed densely defined linear operator with a densely-defined inverse $L^{-1}$. Automatically $L^{-1}$ is closed because $L$ is closed (their graphs are transposes of each other.) That means that $L^{-1}$ will have a closed densely-defined adjoint $(L^{-1})^{\star}$. As you might guess, $(L^{-1})^{\star}=(L^{\star})^{-1}$.
To show $(L^{-1})^{\star}=(L^{\star})^{-1}$: First, you must show that $L^{\star}$ has an inverse, which comes down to showing $\tau\mathcal{G}(L^{\star})$ is a graph:
$$
\tau\mathcal{G}(L^{\star})=\tau[J\mathcal{G}(L)^{\perp}]=
[J\tau\mathcal{G}(L)]^{\perp}=
[J\mathcal{G}(L^{-1})]^{\perp}=\mathcal{G}((L^{-1})^{\star}).
$$
Obviously the subspace on the far right is a graph. So $L^{\star}$ has an inverse and $(L^{\star})^{-1}=(L^{-1})^{\star}$. This proves the following:
Lemma: Let $H$ be a Hilbert space and $L$ a densely-defined closed linear operator on $H$. If $L$ has a densely-defined inverse $L^{-1}$, then $L^{\star}$ has a densely-defined inverse, and $(L^{\star})^{-1}=(L^{-1})^{\star}$.
Note: If $L^{-1}$ is defined everywhere then it is bounded by the closed graph theorem. In that case $(L^{\star})^{-1}$ is also defined everywhere and is bounded because of the graph equation stated in the lemma. This is the case in your problem for $L=U-\lambda I$ because resolvents are, by definition, defined everywhere and are bounded.
Added in Response to your Addition: Another big fact. If $A$ is closed and densely-defined then $A^{\star\star}=A$. This is because $(\mathcal{M}^{\perp})^{\perp}=\mathcal{M}$ for a closed subspace of a Hilbert space such as $H\times H$. This extends the previous lemma.
Lemma: Let $L$ be a closed densely-defined linear operator on a Hilbert space $H$ with adjoint $L^{\star}$. Then $L^{-1}$ exists as a densely-defined linear operator iff $(L^{\star})^{-1}$ exists as a densely-defined linear operator and, in either case, $(L^{-1})^{\star}=(L^{\star})^{-1}$.
Proof: I showed you that if $L$ has a densely-defined inverse, then $L^{\star}$ has a densely defined inverse and $(L^{-1})^{\star}=(L^{\star})^{-1}$. Conversely, if $L^{\star}$ has a densely-defined inverse, then $(L^{\star})^{\star}=L$ has a densely-defined inverse and $((L^{\star})^{-1})^{\star}=L^{-1} \implies (L^{\star})^{-1}=(L^{-1})^{\star}$. $\;\;\Box$
As a corollary: If $L$ is a closed densely-defined linear operator on a Hilbert space, then $L-\lambda I$ has a densely-defined inverse iff $L^{\star}-\overline{\lambda}I$ has a densely-defined inverse and, in either case,
$$
((L-\lambda I)^{-1})^{\star}=(L^{\star}-\overline{\lambda}I)^{-1}.
$$
In particular, $\lambda \in\rho(L)$ iff $\overline{\lambda}\in\rho(L^{\star})$ and the resolvents satisfy $R_{L}(\lambda)^{\star}=R_{L^{\star}}(\overline{\lambda})$. This last equation holds very generally in the sense that one exists iff the other does and, in that case, the two always equal. So, if $L=L^{\star}$ then $\lambda\in\rho(L)$ iff $\overline{\lambda}\in\rho(L)$ and, in that case, $R_{L}(\lambda)^{\star}=R_{L}(\overline{\lambda})$.
Best Answer
You can justify the interchange of orders of integration by letting $$ G(x,y) = e^{-(x-y)^{2}/2} $$ and noticing that $G(x,y)f(x)\overline{g(y)}$ is jointly measurable in $x,y$, and is bounded by \begin{align} |G(x,y)f(x)\overline{g(y)}| & = |G(x,y)^{1/2}f(x)||G(x,y)^{1/2}\overline{g(y)}| \\ & \le \frac{1}{2}G(x,y)|f(x)|^{2}+\frac{1}{2}G(x,y)|g(y)|^{2}. \end{align} So the expression on the left is absolutely integrable on $\mathbb{R}\times\mathbb{R}$ because the expression on the far right is absolutely integrable. Fubini's Theorem now applies to justify the interchange of orders of integration.