Let $t$ denote the antipodal map on $S^n$. There is a simple CW structure for $S^n$ compatible with $t$, which we define inductively by $S^0=\{*,*t\}$, for a point $*$, and $$S^n=S^{n-1}\cup_\phi e_n\cup_\psi e_nt,$$ where $e_n$ is an $n$-cell and $\phi,\psi$ are identifications of the boundaries of $e_n$ and $e_nt$ with $S^{n-1}$.
Intuitively, we regard each sphere as the equator of the next, and glue on a couple of hemispheres.
The boundary maps $d$ are given by $$de_r=e_{r-1}(1+(-1)^rt),\qquad de_0=0.$$
To understand the sign $(-1)^r$ here, note if $r$ is odd then $t$ preserves the orientation of $S^{r-2}$, so the boundary of $e_r$ must be the difference $e_{r-1}-e_{r-1}t$ in order to be closed. Conversely if $r$ is even, then $t$ reverses the orientation of $S^{r-2}$, so the boundary of $e_r$ must be the sum $e_{r-1}+e_{r-1}t$ in order to be closed.
We may take the tensor product of the resulting chain complexes for $S^n$ and $S^m$, to obtain a chain complex for $S^n\times S^m$.
Here the degree $r$ chain group is generated by \begin{eqnarray*}e_a\otimes e_{r-a},\\ e_at\otimes e_{r-a},\\ e_a\otimes e_{r-a}t,\\ e_at\otimes e_{r-a}t.\end{eqnarray*} for $a=0,\cdots, r|a\leq n, r-a\leq m$. The differential is given by \begin{eqnarray*}d(e_i\otimes e_j)&=&e_{i-1} \otimes e_j\\ + &(-1)^i& e_{i-1}t \otimes e_j\\+&(-1)^i& e_{i} \otimes e_{j-1}\\&(-1)^{i+j}& e_{i} \otimes e_{j-1}t.
\end{eqnarray*}
That is we use Leibniz's rule for tensor products:$$d(a\otimes b)=da\otimes b+ (-1)^{{\rm deg}(a)} a\otimes db.$$
Finally, we obtain a chain complex for $$X=(S^n\times S^m)/\langle t\rangle,$$
by taking our chain complex and identifying $$e_i\otimes e_j\sim e_it\otimes e_jt,\qquad e_i\otimes e_jt\sim e_it\otimes e_j.$$
We may write the boundary maps $d$ as matrices, with respect to the basis $\ldots e_a\otimes e_b, e_at\otimes e_b, e_{a+1}\otimes e_{b-1},\ldots$.
The homology groups of $X$ may then be computed as an exercise in linear algebra. For example, if $0<<m<<n$ and $n,m$ both odd I get that the non-zero homology groups are:
\begin{eqnarray*} H_0(X)\cong H_{n+m}(X)&\cong& \mathbb{Z},\\
H_m(X)\cong H_n(X)&\cong&\mathbb{Z},\\
H_{2i-1}(X)\cong H_{n+m-2i}&\cong& \mathbb{Z}/2\mathbb{Z}, \qquad{\rm for} \,\,i=1,\cdots,(m-1)/2.\end{eqnarray*}
I suggest that you do this calculation yourself, as it is quite fiddly and I may have made an error. However the above answer has the symmetries one would expect. If you require help, I can give you the matrices.
Best Answer
Consider the long exact sequence of cohomology. $$\cdots\to H^1(\partial M;\Bbb Z)\to H^2(M,\partial M;\Bbb Z)\overset{d^\ast}{\to} H^2(M;\Bbb Z)\to H^2(\partial M;\Bbb Z)\to\cdots$$ We know that each boundary component is a rational homology $3$-sphere, so $0=H_1(\partial M;\Bbb Q)\cong H_1(\partial M;\Bbb Z)\otimes\Bbb Q$. Therefore, $H_1(\partial M;\Bbb Z)$ consists of torsion only, so $H^1(\partial M;\Bbb Z)=0$ by UCT and the coboundary map $d^*:H^2(M,\partial M)\to H^2(M;\Bbb Z)$ is injective by exactness. Now, UCT implies that $H^2(M;\Bbb Z)\cong \operatorname{Hom}(H_2(M;\Bbb Z),\Bbb Z)\oplus \operatorname{Ext}(H_1(M;\Bbb Z),\Bbb Z)$, which contains no $2$-torsion by assumption. Hence, $H_2(M;\Bbb Z)\cong H^2(M;\partial M;\Bbb Z)$ has no $2$-torsion as it's embedded as a subgroup of $H^2(M;\Bbb Z)$ by $d^\ast$.
[See edit] [The argument only requires $H^1(\partial M;\Bbb Z)=0$, so the condition that the boundary consists of rational homology $3$-spheres is unnecessary.]
Edit:
Actually, saying $\partial M$ is the disjoint union of rational homology $3$-spheres is equivalent to saying $H^1(\partial M;\Bbb Z)=0$. One direction is already demonstrated in the argument above. For the other direction, note that $\partial M$ is a closed, orientable $3$-manifold. If $H^1(\partial M;\Bbb Z)=0$, then $H_1(\partial M;\Bbb Z)$ consists of torsion elements only. By Poincare duality (applied to each component of $\partial M$), we see that $H^2(\partial M;\Bbb Z)\cong H_1(\partial M;\Bbb Z)$ also consists of torsion elements. Apply UCT, we see that $H_2(\partial M;\Bbb Z)$ cannot have free factors, i.e., also consists of torsion elements. Tensor product with $\Bbb Q$ kills all torsion elements, so each connected component is a rational homology $3$-sphere.