This question grew out of this post.
Question: Is there a finitely generated, infinite, residually finite group such that every finite index subgroup has $p$-power index for a fixed prime $p$?
The $p$-adic integers $\mathbb Z_p$ give an example of a non-finitely generated such group. This is not entirely obvious, but follows from a result of Jean-Pierre Serre, which states that every finite index subgroup of $\mathbb Z_p$ is closed.
EDIT: André Henriques has pointed out that Rostislav Grigorchuk constucted a finitely generated, infinite, residually finite $2$-group. All finite quotients of this group have to be $2$-groups. Colin Reid asked in a comment whether there is a torsionfree example. So let me take the freedom to extend my question:
Question: Is there a torsionfree example?
Best Answer
Yes, there are torsion free examples. I do not know who constructed them first, but some examples can be found in papers by Grigorchuk and his co-authors. For example Bartoldi and Grigorchuk proved that a certain Fabrykowski-Gupta group $\Gamma $ has the following properties (see Propositions 6.4 and 6.5 in arXiv:math/9911206):
(a) It is a subgroup of the automorphism group of a rooted tree.
(b) It is virtually torsion free.
(c) It satisfies the following 'congruence property': every finite index subgroup of $\Gamma $ contains a level stabilizer (i.e., the stabilizer of a level of the tree) and the index of every level stabilizer is a power of 3.
By (a) $\Gamma $ is residually finite. By (b) there is a torsion free subgroup $K$ of finite index in $\Gamma $. Since every finite index normal subgroup of $K$ contains a finite index normal subgroup of $\Gamma $, (c) implies that every finite quotient of $K$ is a 3-group.