The relationship between Z and Q is very different from the relationship between R and C.
Not every finite subgroup of GL(2,C) can be embedded in GL(2,R).
Finite subgroups of GL(2,R) have a faithful real character of degree 2 whose irreducible components have Frobenius-Schur index 1, while finite subgroups of GL(2,C) have a faithful character of degree 2.
The group C4 × C4 for instance has no faithful real character of degree 2, so is isomorphic to a subgroup of GL(2,C) but not isomorphic to a subgroup of GL(2,R).
The smallest counterexample is the quaternion group of order 8.
As Mike points out, "smallest" is not a precise terminology. However, since we already have a model $\mathbb{Q}/\mathbb{Z}$, one can prove what razivo suggests at least, i.e. there's no proper subgroups of $\mathbb{Q}/\mathbb{Z}$ satisfying the desired property. That's basically the existing question:Q/Z has a unique subgroup of order n for any positive integer n?. In the process, one actually shows each element of $\mathbb{Q}/\mathbb{Z}$ lies in some cyclic group of order $n$ (Note: $n$ might not be unique though). Thus if any element is taken away, there must be some cyclic group hurt.
PS. The torsion subgroup of $\mathbb{C}^\times$, which lhf suggests, is isomorphic to $\mathbb{Q}/\mathbb{Z}$ through the exponential map.
EDIT: I would like to present a possibly "fair" way to understand "smallest" in this context. For example, we hope that the smallest group containing $\mathbb{Z}/3\mathbb{Z}, \mathbb{Z}/6\mathbb{Z}$ and $\mathbb{Z}/8\mathbb{Z}$ should be $\mathbb{Z}/24\mathbb{Z}$. When we add more groups, for example, $\mathbb{Z}/5\mathbb{Z}$ and $\mathbb{Z}/10\mathbb{Z}$, we could embed all the groups mentioned above into $\mathbb{Z}/120\mathbb{Z}$, which is again smallest. Now it's inspiring to consider the diagram consisting of all cyclic groups $\mathbb{Z}/n\mathbb{Z}$, with arrows the emdeddings from $\mathbb{Z}/d\mathbb{Z}$ into $\mathbb{Z}/n\mathbb{Z}$ if and only if $d|n$. The colimit of this (directed) diagram seems to be the precise characterisation of "smallest" in our sense. Not surprisingly, one can prove this colimit is exactly $\mathbb{Q}/\mathbb{Z}$.
A convenient proof may use the disjoint-union characterisation of directed colimits, based on which we notice: once we identify $\mathbb{Z}/n\mathbb{Z}$ as $\frac{1}{n}\mathbb{Q}/\mathbb{Z}$, all the equivalent relations required in the characterisation of colimits (see the link above) coincide with those in $\mathbb{Q}/\mathbb{Z}$. For example, $4$ in $\mathbb{Z}/10\mathbb{Z}$ is equivalent with $6$ in $\mathbb{Z}/15\mathbb{Z}$, since they both correspond to $12$ in $\mathbb{Z}/30\mathbb{Z}$. This corresponds exactly to the fact $\frac{4}{10} = \frac{6}{15}$ in $\mathbb{Q}/\mathbb{Z}$ (since they both equal to $\frac{12}{30}$, by elementary arithmetic). Thus the directed colimit coincides with $\mathbb{Q}/\mathbb{Z}$.
Best Answer
Yes, $G$ contains only finitely many finite subgroups. As you pointed out, this is trivial if $G/N$ is finite, so we will consider the case $G/N \cong \mathbb{Z}$.
Let $K$ be a finite subgroup of $G$, then $KN$ is also a finite subgroup of $G$ (since $N$ is finite and normal) and therefore $KN/N$ is a finite subgroup of $G/N \cong \mathbb{Z}$. We conclude that $KN/N = \left\{0\right\}$, hence $KN = N$ and so $K < N$. This means that we have only a finite number of choices for $K$.