The image of a finite subgroup of $\text{SU}(2)$ in $\text{SO}(3)$ is a finite subgroup of $\text{SO}(3)$; moreover, the kernel is either trivial or $\{ \pm 1 \}$. But $-1$ is the unique element of order $2$ in $\text{SU}(2)$, so any group of even order contains it.
I claim all the finite subgroups of odd order are cyclic. This follows because the inclusion $G \to \text{SU}(2)$ cannot define an irreducible representation of $G$ (since otherwise $2 | |G|$), hence it must break up into a direct sum of dual $1$-dimensional representations.
So once you know the finite subgroups of $\text{SO}(3)$, you already know the finite subgroups of $\text{SU}(2)$.
Here is a proof that no finite group has faithful irreducible representations of degrees $2$ and $3$. It might be easier to do this using the classification, of finite subgroups of ${\rm G}(2,{\mathbb C})$, but I am more familiar with the subgroups of ${\rm GL}(2,q)$ for finite $q$, so I will use that approach.
Let $G$ be a finite irreducible subgroup of ${\rm GL}(2,{\mathbb C})$. Let $p$ be a prime not dividing $|G|$. Then, by standard results in representation theory, the associated complex representation can be written over the finite field of order $q$ for some power $q$ of $p$.
The irreducible subgroups of ${\rm GL}(2,q)$ are either imprimitive or semilinear, or they have normal subgroups isomorphic to ${\rm SL}(2,3)$ or ${\rm SL}(2,5)$.
The imprimitive and semilinear groups have a normal abelian subgroup of index $2$, and since all complex irreducible representations of abelian groups have degree $1$, the irreducible representations of $G$ have degree at most $2$.
On the other hand, ${\rm SL}(2,3)$ or ${\rm SL}(2,5)$ do not have faithful irreducible representations of degree $3$.
Best Answer
Let $ G $ be a finite subgroup of $ Sp_2 $. That is equivalent to being a finite subgroup of $ GL_2(\mathbb{H}) $ the group of $ 2 \times 2 $ invertible quaternionic matrices. The finite subgroups of $ GL_2(\mathbb{H}) $ are classified in
https://core.ac.uk/download/pdf/82740228.pdf
There are a bunch of solvable subgroups, possibilities given on pages 477-478.
The quasisimple ones are $ SL_2(5) $ and $ SL_2(9) $.
Another interesting non solvable subgroup is $ \mathcal{B}^* $, the lift through the double cover $ Sp_2 \to SO_5 $ of the weyl group $ W(D_5) $ of determinant 1 signed permutation matrices. Also $ \mathcal{B} $, the commutator subgroup of $ \mathcal{B}^* $, is PerfectGroup(1920,6).
There are some variants on those groups also, but those are basically the most interesting ones.