I have computed that the 1-form $\omega:= p^*\omega' \in \Omega^1(Q\times H', \mathfrak{h'})$ descends to a form on $Q\times_{\alpha}H'$
I don't think this is true: firstly as Ted points out, the Lie group homomorphism $\alpha:H\to H'$ induces a Lie algebra homomorphism $\dot{\alpha} = T_e\alpha:\mathfrak{h}\to\mathfrak{h}'$. For $(q,h')\in Q\times H'$ and $\xi\in\mathfrak{h}$, the vector along the quotient direction
$$
(q\cdot \xi, -\dot\alpha(\xi)\cdot h')\in T_{(q,h')}(Q\times H')
$$
when contracted with $\omega = p^*\omega'$ takes the value $-\dot\alpha(\xi)$, which is non-zero for $\xi\notin \ker \dot\alpha$. However this vector descends to the zero vector on $Q\times_\alpha H'$.
I haven't worked through the details, but I think the correct form to consider would be
$$
\omega = \dot\alpha(p_1^*\omega_Q) + p_2^*\omega',
$$
where $p_1:Q\times H'\to Q$ and $p_2:Q\times H'\to H'$ denote the respective projections. It doesn't suffer from the above problem: applying it to the quotient direction $(q\cdot \xi, -\dot\alpha(\xi)\cdot h')$ yields 0, and so it should descend to a 1-form on $Q\times_\alpha H'$, which will be a connection.
Edit: I went back to work out the details of this, and it seems I was incorrect in my guess. The correct form should be
$$
\omega_{(q,h')} :=\mathrm{Ad}_{(h')^{-1}}\left(\dot{\alpha}(p_1^*\omega_Q )_{(q,h')}\right) + (p_2^*\omega_L')_{(q,h')}=\mathrm{Ad}_{(h')^{-1}}\left(\dot{\alpha}(p_1^*\omega_Q )_{(q,h')} + (p_2^*\omega_R')_{(q,h')}\right)
$$
Here $\omega_L' (\omega_R')$ is the left-(right-)invariant Maurer-Cartan form on $H'$. Letting $\pi_\alpha:Q\times H' \to Q\times_\alpha H'$ denote the quotient map,
$\omega$ vanishes on vectors of the form $(q\cdot\xi, -\dot\alpha(\xi)\cdot h')$ (i.e. vectors along the fibres of $\pi_\alpha$), and is invariant under the quotient action. So it reduces to a form $\omega_\alpha$ on $Q\times_\alpha H'$
$$
\omega = \pi_\alpha^*\omega_\alpha.
$$
The obvious right actions $R_\cdot$ and $\check{R}_\cdot$ of $H'$ on $Q\times H'$ and $Q\times_\alpha H'$ commute with $\pi_\alpha$,
$$
\pi_\alpha\circ R_{h'} = \check{R}_{h'}\circ \pi_\alpha,
$$
and it is straightforward to verify that $R_{k'}^*\omega = \mathrm{Ad}_{(k')^{-1}}\omega$ for all $k'\in H'$. It follows that
$$
R_{k'}^*\pi_\alpha^*\omega_\alpha = \mathrm{Ad}_{(k')^{-1}}\pi_\alpha^*\omega_\alpha \implies \pi_\alpha^*\check{R^*}_{k'}\omega_\alpha = \pi_\alpha^*(\mathrm{Ad}_{(k')^{-1}}\omega_\alpha).
$$
Since $\pi_\alpha$ is a submersion, this implies $\check{R^*}_{k'}\omega_\alpha = \mathrm{Ad}_{(k')^{-1}}\omega_\alpha$. Also $\omega_{(q,h)}((q,h)\cdot \xi) =\omega_{(q,h)}(0_q,h\cdot \xi) = \xi$ and the fact that $\pi_\alpha$ commutes with the right action implies the equivalent result for $\omega_\alpha$. So $\omega_\alpha$ is a connection form on $Q\times_\alpha H'$.
Best Answer
As noted in the comment by @AndrewDHwang, the fact that $B$ is symplectic is not relevant here, this is just general principal bundle theory. Writing the exponential map of $S^1=U(1)$ as $t\mapsto e^{it}$, you define $R(x):=\tfrac{d}{dt}|_{t=0}x\cdot e^{it}$, where in the right hand side you use the principal right action $M\times S^1\to M$. Then obviously the flow lines of $R$ are the orbits of the $S^1$-action. Morover, by defintion, $R$ is the fundamental vector field corresponding to a $1\in\mathbb R$ where we view $\mathbb R$ as the Lie algebra of $S^1$.
Thus any fundamental vector field can be written as $\lambda R$ for $\lambda\in\mathbb R$ and the fact that $\alpha$ reproduces the generators of fundamental vector fields hence is equivalent to $\alpha(R)=1$. Now $\alpha$ is a principal connection form if in addition it is equivariant for the principal right action and this is easily seen to be equivalent to $\mathcal L_R\alpha=0$.