Over spheres, you actually have a nice way to distinguish bundles.
Every sphere $S^n$ can be written as a union of two contractible spaces, $U \cup V$, whose intersection is a small neighborhood of the equator, $\mathbb{R} \times S^{n-1}$. For instance if $x,y$ are the north and south poles, respectively, you can take $U = S^n - \{x\}, V = X^n - \{y\}$.
Over $U$ and $V$, any bundle is trivial, as you noted. The bundle on the sphere is glued together from these two trivial bundles.
If you're interested in a vector bundle, the transition function for the bundle is given by a map $U \cap V \to GL_m$, for a vector bundle of rank $m$. If this map is null-homotopic, meaning that it can be homotoped to the constant map, then your bundle is trivial, as any trivialization over $U$ can then be glued to a trivialization over $V$. So in the end you are reduced to the study of maps from $U \cap V \simeq S^{n-1}$ to your structure group -- i.e., $\pi_{n-1}GL_m$.
For $m=1$, this amounts to asking whether any map $S^{n-1} \to GL_1$ is null-homotopic. In your case for $S^3$, since $GL_1(\mathbb{R})$ is a disjoint union of two contractible spaces, and $S^2$ is connected, any map is null-homotopic.
In the fiber bundle case with fiber $S^1$ and base $S^3$, you are looking at maps $S^2 \to S^1$. But $\pi_2(S^1) = *$ so any circle bundle is contractible over $S^3$.
Note that both facts hold for higher spheres as well.
This is a partial answer.
For any abelian group $G$ and integer $n \geq 1$, there is a CW complex $M(G, n)$ such that
$$\tilde{H}_i(M(G, n)) = \begin{cases}
G & i = n\\
0 & i \neq n
\end{cases}$$
called a Moore space. Moreover, for $n > 1$, we can take these spaces to be simply connected. See Example $2.40$ of Hatcher's Algebraic Topology.
Using the fact that $H_n(\bigvee_{\alpha}X_{\alpha}) = \bigoplus_{\alpha}H_n(X_{\alpha})$, we see that for a given sequence of abelian groups $\{G_n\}_{n=1}^{\infty}$, the CW complex $X = \bigvee_{n=1}^{\infty}M(G_n, n)$ has the property that $H_i(X) = G_i$.
However, all that can be said of $\pi_1(X)$ is that its abelianisation is $G_1$. There are many different groups with the same abelianisation, so $\pi_1(X)$ may not be the group you wanted it to be.
If one could construct the Moore spaces $M(G, 1)$ with a given fundamental group (which must satisfy the necessary condition that its abelianisation is $G$), then the above space would have the desired properties. However, I don't know if this has been done.
Added Later: I asked a separate question about this final point (whether one could construct a Moore space $M(G, 1)$ with given fundamental group) here. It turns out it is not always possible.
Best Answer
In light of the fact that I've made multiple mistakes whilst thinking this through, I will try to write a somewhat thorough answer, so bear with me. Let $S^1\rightarrow M\stackrel{\pi}{\rightarrow}B$ be an orientable circle bundle whose base is a path-connected CW-complex. (I am assuming this for the sake of simplicity, since you seem to care for smooth manifolds. The results can be generalized to numerable bundles whose base has the homotopy type of a CW-complex.) Fix basepoints throughout, though I will omit them from notation, and a universal cover $p\colon\tilde{B}\rightarrow B$ of $B$. I claim that $\pi_1(M)$ is finite if and only if $\pi_1(B)$ is finite and the pullback $S^1\rightarrow p^{\ast}(M)\stackrel{p^{\ast}\pi}{\rightarrow}\tilde{B}$ is a non-trivial bundle. This is the case if and only if $\pi_1(B)$ is finite and the Euler class $u({\pi})\in H^2(B)$ is not in the kernel of $p^{\ast}\colon H^2(B)\rightarrow H^2(\tilde{B})$. Two notable corollaries:
If $S^1\rightarrow M\stackrel{\pi}{\rightarrow}B$ is a non-trivial, orientable circle bundle over a simply connected CW-complex, then $\pi_1(M)$ is finite.
If $S^1\rightarrow M\stackrel{\pi}{\rightarrow}B$ is an orientable circle bundle over a path-connected CW-complex and $u(\pi)\in H^2(B)$ is torsion, then $\pi_1(M)$ is infinite.
The orientability assumption says that the bundle has structure group $\mathrm{Homeo}^+(S^1)$, the orientation-preserving homeomorphisms of $S^1$, hence is classified by a map (unique up to homotopy) $B\rightarrow B\mathrm{Homeo}_+(S^1)$. Note that $\mathrm{Homeo}^+(S^1)\cong\mathrm{Homeo}^+_e(S^1)\times S^1$, where $\mathrm{Homeo}_e^+(S^1)$ is the group of orientation-preserving homeomorphisms of $S^1$ fixing the identity. This group is isomorphic to the group $\mathrm{Homeo}_{\partial I}(I)$ of homeomorphisms of $I$ that fix the boundary point-wise, which is contractible by a straight-line homotopy. Thus, $B\mathrm{Homeo}^+(S^1)\simeq B\mathrm{Homeo}^+_e(S^1)\times BS^1\simeq BS^1\simeq\mathbb{CP}^{\infty}$ (the last homotopy equivalence due to the explicit model $S^1\rightarrow S^{\infty}\rightarrow\mathbb{CP}^{\infty}$ of a universal principal $S^1$-bundle). However, $\mathbb{CP}^{\infty}$ is a $K(\mathbb{Z},2)$, so ultimately the bundle is classified by an element of $[B,B\mathrm{Homeo}^+(S^1)]=[B,\mathbb{CP}^{\infty}]=H^2(B)$, a characteristic class $u(\pi)$ called the Euler class. More precisely, the universal element $u\in H^2(\mathbb{CP}^{\infty})$ that classifies the universal bundle is a generator and if $S^1\rightarrow M\stackrel{\pi}{\rightarrow}B$ is classified by $f\colon B\rightarrow\mathbb{CP}^{\infty}$, then $u(\pi)=f^{\ast}(u)$.
To study $\pi_1(M)$, our primary tool is the LES of a fibration, which reads $$\require{AMScd} \begin{CD} \dotsc @>>> \pi_2(B) @>{\partial}>> \pi_1(S^1) @>>> \pi_1(M) @>>> \pi_1(B) @>>> 1. \end{CD}$$ Thus, $\pi_1(M)$ is an extension of $\pi_1(B)$ by $\mathrm{coker}(\pi_2(B)\rightarrow\pi_1(S^1))$, hence $\pi_1(M)$ is finite if and only if $\pi_1(B)$ and $\mathrm{coker}(\pi_2(B)\rightarrow\pi_1(S^1))$ are finite. The group $\pi_1(S^1)\cong\mathbb{Z}$ is infinite, yet all of its non-trivial quotients are finite. Thus, $\pi_1(M)$ is finite if and only if $\pi_1(B)$ is finite and $\partial\colon\pi_2(B)\rightarrow\pi_1(S^1)$ is non-trivial. This latter condition can be analyzed further. To do this, let me set up two commutative diagrams, namely $$\require{AMScd} \begin{CD} \dotsc @>>> \pi_2(\mathbb{CP}^{\infty}) @>{\partial}>> \pi_1(S^1) @>>> \pi_1(\mathbb{C}^{\infty})=1 @>>> \pi_1(\mathbb{CP}^{\infty})=1 @>>> 1\\ {} @AAA @A{\mathrm{id}}AA @AAA @AAA\\ \dotsc @>>> \pi_2(B) @>{\partial}>> \pi_1(S^1) @>>> \pi_1(M) @>>> \pi_1(B) @>>> 1\\ {} @A{\wr}AA @A{\mathrm{id}}AA @AAA @AAA\\ \dotsc @>>> \pi_2(\tilde{B}) @>{\partial}>> \pi_1(S^1) @>>> \pi_1(p^{\ast}M) @>>> \pi_1(\tilde{B})=1 @>>> 1 \end{CD}$$ and $$\require{AMScd} \begin{CD} H^2(\tilde{B}) @>{\sim}>> \mathrm{Hom}(H_2(\tilde{B}),\mathbb{Z}) @>{\sim}>> \mathrm{Hom}(\pi_2(\tilde{B}),\mathbb{Z})\\ @AAA @AAA @A{\wr}AA\\ H^2(B) @>>> \mathrm{Hom}(H_2(B),\mathbb{Z}) @>>> \mathrm{Hom}(\pi_2(B),\mathbb{Z})\\ @AAA @AAA @AAA\\ H^2(\mathbb{CP}^{\infty}) @>{\sim}>> \mathrm{Hom}(H_2(\mathbb{CP}^{\infty}),\mathbb{Z}) @>{\sim}>> \mathrm{Hom}(\pi_2(\mathbb{CP}^{\infty}),\mathbb{Z}). \end{CD}$$ They are both induced by the universal covering $\tilde{B}\rightarrow B$ and the classifying map $B\rightarrow\mathbb{CP}^{\infty}$. The former is the naturality diagram comparing the LESs of the respective bundles. The latter is the naturality diagram comparing the respective Hurewicz maps. The horizontal isomorphisms are due to the universal coefficient and Hurewicz theorems, since $\tilde{B}$ and $\mathbb{CP}^{\infty}$ are simply connected. The vertical isomorphisms are due to the fact that a universal cover induces isomorphisms on higher homotopy groups.
Identifying $\pi_1(S^1)=\mathbb{Z}$, the boundary maps in the respective LESs in diagram $1$ are identified with elements of the $\mathrm{Hom}$-groups in the right column of diagram $2$. The commutativity of diagram $1$ implies that these elements map to one another under the vertical arrows in the right column of diagram $2$. The boundary map of the universal bundle is an isomorphism (to its left comes $\pi_2(\mathbb{C}^{\infty})=1$), so it generates $\mathrm{Hom}(\pi_2(\mathbb{CP}^{\infty}),\mathbb{Z})$ and the isomorphisms in the bottom row of diagram $2$ imply that it is, up to sign depending on the chosen generators, the image of the universal element $u\in H^2(\mathbb{CP}^{\infty})$. The lower part of diagram $2$ then implies that the Euler class $u(\pi)\in H^2(B)$ maps to the boundary map $\partial\in\mathrm{Hom}(\pi_2(B),\mathbb{Z})$, perhaps up to sign. The isomorphisms in the upper part of diagram $2$ then imply that this element is trivial if and only if $u(\pi)$ is in the kernel of $p^{\ast}\colon H^2(B)\rightarrow H^2(\tilde{B})$. However, naturality implies that $p^{\ast}(u(\pi))=u(p^{\ast}\pi)$, so this is the case if and only if the pullback bundle $p^{\ast}\pi\colon p^{\ast}M\rightarrow\tilde{B}$ is trivial.
Edit: I claim furthermore that, if $\pi_1(B)$ is finite, the kernel of $H^2(B)\rightarrow H^2(\tilde{B})$ is precisely the torsion subgroup $TH^2(B)$ of $H^2(B)$. To see this, consider the natural diagram of SESs given by the universal coefficient theorem, $$\require{AMScd} \begin{CD} 0 @>>> 0 @>>> H^2(\tilde{B}) @>{\sim}>> \mathrm{Hom}(H_2(\tilde{B}),\mathbb{Z}) @>>> 0\\ {} @AAA @AAA @AAA\\ 0 @>>> \mathrm{Ext}(H_1(B),\mathbb{Z}) @>>> H^2(B) @>>> \mathrm{Hom}(H_2(B),\mathbb{Z}) @>>> 0. \end{CD}$$ Since $H_1(B)=\pi_1(B)^{\mathrm{ab}}$ is finite, $\mathrm{Ext}(H_1(B),\mathbb{Z})$ is torsion. Evidently, $\mathrm{Hom}(H_2(B),\mathbb{Z})$ is torsion-free, so it follows from exactness that $\mathrm{Ext}(H_1(B),\mathbb{Z})$ is the torsion subgroup of $H^2(B)$. The claim is now, by a short diagram chase, equivalent to the injectivity of $\mathrm{Hom}(H_2(B),\mathbb{Z})\rightarrow\mathrm{Hom}(H_2(\tilde{B}),\mathbb{Z})$. This map is dual to the map $H_2(\tilde{B})\rightarrow H_2(B)$, which fits into the commutative square $$\require{AMScd} \begin{CD} \pi_2(\tilde{B}) @>{\sim}>> H_2(\tilde{B})\\ @V{\wr}VV @VVV\\ \pi_2(B) @>>> H_2(B). \end{CD}$$ The horizontal maps are Hurewicz homomorphisms. Thus, the cokernel $C$ of $H_2(\tilde{B})\rightarrow H_2(B)$ is the cokernel of $\pi_2(B)\rightarrow H_2(B)$, which is isomorphic to $H_2(K(\pi_1(B),1))$ (see Hatcher, p.390/391). This group is finite since $\pi_1(B)$ is finite (it is finitely generated since $K(\pi_1(B),1)$ can be realized as a simplicial complex with finitely many simplices in each dimension, so its homology is computed by a chain complex of finitely generated abelian groups, and it has no free summand cause the rational homology vanishes, because the rational homology of its universal cover vanishes (see e.g. here) as that universal cover is contractible). Now, applying the left-exact functor $\mathrm{Hom}(-,\mathbb{Z})$ transforms the half-exact sequence $H_2(\tilde{B})\rightarrow H_2(B)\rightarrow C\rightarrow 0$ into the half-exact sequence $0\rightarrow\mathrm{Hom}(C,\mathbb{Z})=0\rightarrow\mathrm{Hom}(H_2(B),\mathbb{Z})\rightarrow\mathrm{Hom}(H_2(\tilde{B}),\mathbb{Z})$. Thus, the map is injective, as desired.