Using polarization, Ito's isometry implies $ \mathbb{E}\left( \left( \int_0^t f(s) \mathrm{d} W_s \right) \left( \int_0^t g(s) \mathrm{d} W_s \right) \right) = \int_0^t f(s) g(s) \mathrm{d} s$.
Thus, for $s<t$
$$ \begin{eqnarray}
\mathbb{E} \left( X_s X_t \right) &=&
\mathbb{E} \left( \left( \sigma \int_0^t \mathrm{e}^{-a(s - \tau)} \mathbf{1}\left(\tau \le s\right) \mathrm{d}W_\tau \right) \left( \sigma \int_0^t \mathrm{e}^{-a(t - \tau)} \mathrm{d}W_\tau \right) \right) \\ &=&
\sigma^2 \int_0^{\min(s,t)} \mathrm{e}^{-a (s+t-2 \tau)}\mathrm{d} \tau =
\sigma^2 e^{-a (s+t)} \cdot \frac{e^{2 a \min (s,t)}-1}{2 a} =
\sigma^2 e^{-a t} \cdot \frac{\sinh(a s)}{a}
\end{eqnarray}
$$
Using $W_s = \int_0^s 1 \,\, \mathrm{d} W_\tau$, we similarly get
$$
\mathbb{E} \left( X_t W_s\right) = \sigma \int_0^s \mathrm{e}^{-a(t-\tau)}\mathrm{d} \tau
= \sigma \mathrm{e}^{-a t} \cdot \frac{ \left(\mathrm{e}^{a s}-1\right)}{a}
$$
If we apply Itô's formula to the function
$$f(x) := \exp(\lambda x)$$
and the Itô process $(Y_t)_{t \geq 0}$, then we find
$$e^{\lambda Y_t}-1 = \lambda \int_0^t e^{\lambda Y_s} \alpha_s \, dW_s + \frac{\lambda^2}{2} \int_0^t e^{\lambda Y_s} \alpha^2(s) \, ds.$$
Since the first term on the right-hand side is a martingale, we get for $\phi_{\lambda}(t):= \mathbb{E}e^{\lambda Y_t}$
$$\phi_{\lambda}(t) -1 = \frac{\lambda^2}{2} \int_0^t \phi_{\lambda}(s) \alpha^2(s) \, ds.$$
This ordinary differential equation can be solved explicitely,
$$\mathbb{E}e^{\lambda Y_t} = \phi_{\lambda}(t) = \exp \left( \frac{\lambda^2}{2} \int_0^t \alpha(s)^2 \, ds \right).$$
This proves that $Y_t$ is normal with mean $0$ and variance $\int_0^t \alpha(s)^2 \, ds$.
Edit As @NateEldredge pointed out, we have to ensure that $(e^{\lambda Y_s})_{s \geq 0}$ is suitable integrable; for a proof that this is indeed the case see e.g. René L. Schilling/Lothar Partzsch: Brownian motion - An introduction to stochastic processes, Chapter 18 (2nd edition) or my other answer.
Best Answer
The expectation of the Ito integral $\mathbb{E}( \int_0^t \mathrm{e}^{a s} \mathrm{d} W_s )$ is zero as George already said.
To compute $\mathbb{E}( W_t \int_0^t \mathrm{e}^{a s} \mathrm{d} W_s )$, write $W_t = \int_0^t \mathrm{d} W_s$. Then use Ito isometry:
$$ \mathbb{E}( W_t \int_0^t \mathrm{e}^{a s} \mathrm{d} W_s ) = \mathbb{E}\left( \int_0^t \mathrm{d} W_s \cdot \int_0^t \mathrm{e}^{a s} \mathrm{d} W_s \right) = \int_0^t (1 \cdot \mathrm{e}^{a s}) \mathrm{d} s = \frac{\mathrm{e}^{a t} - 1}{a} \phantom{hhhh} $$