Let $p$ be a prime. Prove that the group $\langle x,y\mid x^p=y^p=(xy)^p=1\rangle$ is infinite if $p>2$, but that if $p=2$, it is a Klein 4-group.
Let $G$ be the group in the problem statement. The case $p=2$ I could prove it. Let $V=\langle a,b\rangle$ be the 4-group, $F$ free on $X=\{x,y\}, f:X\to V, x\mapsto a, y\mapsto b$. Then there exists $\varphi:F\to V$ such that $\varphi\mid X=f$. $\varphi$ is onto because $V=\langle a,b\rangle$. Let $\Delta=\{x^2,y^2,(xy)^2\}$. Then $x^2\varphi=a^2=1, y^2\varphi=b^2=1, (xy)^2\varphi=[(xy)\varphi]^2=(ab)^2=1$. So $\Delta \subseteq$ ker $\varphi$ and $R=\Delta^F\le$ ker $\varphi\le F$. By the third isomorphism theorem, there is an epimorphism $\phi:F/R\to F/$ker $\varphi$. But $F/$ker $\varphi \simeq V$. So we have an epimorphism $\theta:G=F/R\to V$ and $\mid G\mid \ge\mid V\mid =4$.
On the other hand, $xyxy=(xy)^2=1, yx=x^{-1}y^{-1}=xy$. So every element in $G$ can be written as $x^i y^j, 0 \le i,j\lt 2$, stictly speaking as $x^i y^j R$. So $\mid G\mid \le 4$ and $\mid G\mid = 4$. So we have that $V$ is generated by two elements satisfying the same relations as in $G$ and that the two groups have the same order, proving that $G\simeq V$.
The case $p=3$ has been posted here: Presentation $\langle x,y \mid x^3=y^3=(xy)^3=1\rangle\cong\langle t\rangle\ltimes A$ I think the present problem is a generalization of this problem. That is the problem in the link is a particular case of the title problem. So for the general case I could try to find a normal abelian subgroup as in the hint to the problem in the link. To begin with, I could look for two words in $F$ that commute. I tried with $\langle xyx,x^{p-1}y\rangle$ but I failed. I think the problem is really difficult. Could you give me a hint?
Best Answer
The following proof will work for any $p \ge 3$, not necessarily prime.
Let $\omega$ be a primitive complex $p$-th root of $1$, and define the complex matrices $$a = \left(\begin{array}{ccc}\omega&0&0\\0&\omega^{-1}&0\\0&0&1\end{array}\right),\ \ \ \ b = \left(\begin{array}{ccc}\omega&0&0\\0&\omega^{-1}&0\\1&0&1\end{array}\right).$$ Then clearly $a^p=I$ and, since $b$ has distinct eigenvalues, it is similar to a diagonal matrix with the same diagonal entries, and hence $b^p=I$. Similarly $(ab)^p=1$. But $$a^{-1}b = \left(\begin{array}{ccc}1&0&0\\0&1&0\\1&0&1\end{array}\right)$$ has infinite order, and so the subgroup of ${\rm GL}(3,{\mathbb C})$ generated by $a$ and $b$ is infinite.
Since $\langle a,b \rangle$ is an epimorphic image of $\langle x,y \mid x^p=y^p=(xy)^p=1 \rangle$, this group must also be infinite.