The index of $A\ast(ba)B(ba)^{-1}$ in $A\ast B$

co-hopfianfree-productgroup-theory

A group is called co-Hopfian if it is not isomorphic to a proper subgroup of itself, while it is called finitely co-Hopfian if it is not isomorphic to a proper finite index subgroup of itself. A co-Hopfian group is finitely co-Hopfian, but the converse is not true (as the examples below will illustrate).

Let $A$ and $B$ be non-trivial groups. As I learnt from this answer, the free product $A\ast B$ is not co-Hopfian, i.e. there is a proper subgroup of $A\ast B$ which is isomorphic to $A\ast B$, namely $A\ast(ba)B(ba)^{-1}$ where $a \in A$ and $b \in B$ are non-identity elements.

What is the index of $A\ast(ba)B(ba)^{-1}$ in $A\ast B$?

I suspect that the index is always infinite. This is the case if $A\ast B$ is finitely co-Hopfian. Here are two such examples:

  1. If $A$ and $B$ are free groups on $m$ and $n$ generators respectively, then $A\ast B$ is a free group on $r := m + n$ generators. Since $A$ and $B$ are non-trivial, we have $m, n \geq 1$ and hence $r \geq 2$. An index $k$ subgroup of a free group on $r$ generators is a free group on $k(r – 1) + 1$ generators, see here for example. If $k > 1$, then $k(r – 1) + 1 > r$, so $A\ast B$ is finitely co-Hopfian.

  2. If $A$ and $B$ are non-trivial finite groups and not both $\mathbb{Z}_2$, then the Euler characteristic of $A\ast B$ is $\chi(A\ast B) = \frac{1}{A} + \frac{1}{B} – 1 \leq \frac{1}{2} + \frac{1}{3} – 1 = -\frac{1}{6}$, so $\chi(A\ast B) \neq 0$. Since the Euler characteristic is multiplicative under finite index subgroups, we see that such a group is finitely co-Hopfian.

The previous argument doesn't apply to $\mathbb{Z}_2\ast\mathbb{Z}_2$ since $\chi(\mathbb{Z}_2\ast\mathbb{Z}_2) = \frac{1}{2} + \frac{1}{2} – 1 = 0$. In fact, the group $\mathbb{Z}_2\ast\mathbb{Z}_2$ is not finitely co-Hopfian as it contains an index two subgroup isomorphic to itself, namely $\mathbb{Z}_2\ast a\mathbb{Z}_2a^{-1}$. This still leaves open the possibility that $\mathbb{Z}_2\ast(ba)\mathbb{Z}_2(ba)^{-1}$ has infinite index though.

My only other observation is that $A\ast(ba)B(ba)^{-1}$ is not a normal subgroup of $A\ast B$: for $h = (ba)b(ba)^{-1} \in A\ast(ba)B(ba)^{-1}$, and $g = (ba)^{-1} \in A\ast B$, we have $ghg^{-1} = (ba)^{-1}(ba)b(ba)^{-1}(ba) = b \not\in A\ast(ba)B(ba)^{-1}$.

Best Answer

Converting comments to answers.

Let $H=A\ast(ba)B(ba)^{-1}\le A\ast B$ be your subgroup. If $|B|>2$, then there's an element $c\in B\setminus\{1,b\}$. The cosets \begin{equation*} H(ca), H(ca)^2,H(ca)^3,\cdots \end{equation*} are all distinct. Indeed, equality $H(ca)^n=H(ca)^m$ for $n>m$ would mean $(ca)^{n-m}\in H$, but this is impossible, as all words in $H$ begin with $b$ or $a$, not $c$. Thus $H$ is infinite index.

If instead $|A|>2$, we can conjugate to show $(ba)^{-1}H(ba)$ is infinite index via the same argument above. Then of course $H$ is infinite index as well.

The case when $|A|=|B|=2$, which is the infinite dihedral group, is the sole counterexample. This is because in this case $A\ast B$ has an infinite cyclic subgroup $K$, which is normal and of index $2$. Since $|H|>2$, it must have nontrivial intersection $H\cap K$. But all nontrivial subgroups of $K$ are finite index, and thus $H\cap K$ is finite index in $K$, hence in $A\ast B$.