I am trying to determine whether the literature contains a complete proof of the classification of finite 2-transitive groups. This is a fundamental result with important applications in many areas of mathematics, so it seems worthwhile to make sure that it has actually been proved. In short, the issue is that it is common to credit Hering for classifying the nonsolvable affine 2-transitive groups, but it seems that Hering never published a paper containing such a classification; and the one paper which does purport to contain a proof, namely a 1985 paper by Liebeck, begins with a logical leap which I cannot follow. [Added later: thanks to Michael Giudici's answer, I can now follow the first step in Liebeck's proof. I have not yet gone through the rest of the proof, but do not presently have any reason to doubt its validity.]
Let me summarize the literature to clarify the issue. By Burnside's classical result, a (finite) 2-transitive group $G$ satisfies one of
- $L\le G\le\text{Aut}(L)$ for some nonabelian simple group $L$
- $G\le\text{AGL}_r(p)$ in the usual action on $\mathbf{F}_p^r$, where in addition $G$ contains all translations by elements of $\mathbf{F}_p^r$.
The possibilities in case 1 were determined by Curtis-Kantor-Seitz 1976 in case $L$ is a Chevalley group of ordinary or twisted type (full citations are at the end of this question, in chronological order). It is easy to resolve the case $L=A_n$ (some people cite Maillet 1895 for this, but I haven't read that paper). When $L$ is sporadic, Cameron 1981 and Kantor 1985 stated the result without proof, and Praeger-Soicher 1997 published a proof (summarized in their Table 5.1). This completes the proof in case 1.
So the question is case $2$, when $G$ is a group of affine transformations of a vector space with $G$ containing all translations, or in short, $G$ is an "affine group". It seems that the first reference claiming that $2$-transitive affine groups had been classified is Cameron 1981, which does not indicate what the groups actually are, but instead merely states as Theorem 5.3 that "All finite 2-transitive groups are known." More informative statements of the result appear in Huppert-Blackburn 1982, Kantor 1985, Abhyankar 1992, Dixon-Mortimer 1996, and Cameron 1999, all of which include lists of groups.
Cameron 1981 says that a paper by Hering (cited as "to appear", but with no title) "examined the known simple groups and determined all such situations in which each could occur as such a composition factor [of an affine 2-transitive group]". I will discuss this unknown paper below.
Huppert-Blackburn 1982 gives a proof in case $G$ is solvable (Thm. XII.7.3; this case was first resolved in Huppert 1957), and then states the full result (without proof) as Remark XII.7.5, attributing it to Hering 1974. But Hering 1974 does not contain the stated result; instead it only determines the 2-transitive affine groups having a composition factor isomorphic to $\text{PSL}_2(q)$ or $A_n$ or the smallest of Janko's four sporadic groups.
Kantor 1985 says that the result follows from Curtis-Kantor-Seitz 1976, Hering 1973, Hering 1974, Hering 1985, Huppert 1957, and Maillet 1895, in addition to further analysis of sporadic groups which is left to the reader. The only papers in Kantor's list which address the affine case are the three papers by Hering, of which the second was summarized above and the other two will be discussed below. (A minor side remark: the statement of the result in Kantor 1985 is actually tautological; one should add the condition $n>1$ in case (B2) to make it have content.)
Abhyankar 1992 (on p.87) cites Cameron 1981, Kantor 1985, Curtis-Kantor-Seitz 1976, and O'Nan 1975. O'Nan 1975 contains wonderful results about 2-transitive groups (impressively, achieved without using the classification of finite simple groups), but does not purport to classify nonsolvable affine 2-transitive groups. Abhyankar does not cite Hering, but also Abhyankar was not a group theorist and did not claim to know the proof, instead saying that the classification of 2-transitive groups "was communicated to me by Cameron".
Dixon-Mortimer 1996 (on p.244) cites Huppert 1957, Hering 1974, and Huppert-Blackburn 1982 for the classification of affine 2-transitive groups. Thus, everything they say about the issue is contained in Huppert-Blackburn 1982.
Cameron 1999 (on p.110) says that the nonsolvable affine 2-transitive groups were determined by Hering; the only paper of Hering's in the bibliography of Cameron's book is Hering 1974, which (on p.194) is claimed to contain precise conditions for which of the groups listed on p.195 of Cameron 1999 are actually 2-transitive (but Hering 1974 does not contain such conditions).
Thus, the group theory community seems united in their belief that Hering has classified the (nonsolvable) affine 2-transitive groups. But I see no evidence that Hering ever published either a statement or a proof of this classification. The only papers of Hering cited by any of the above-mentioned sources are Hering 1973, Hering 1974, and Hering 1985 (as well as the untitled "to appear" paper of Hering's cited in Cameron 1981). I discussed Hering 1974 above: it covers only the case that some composition factor of the group is $\text{PSL}_2(q)$ or $A_n$ or the smallest Janko group. Hering 1985 does not claim to classify nonsolvable affine 2-transitive groups. Instead, the introduction of Hering 1985 says the paper achieves this classification only under the additional assumption that some composition factor of the group is a Chevalley group (of ordinary or twisted type). However, Hering's paper does not contain either a statement or a proof of this promised classification; instead it proves results in this direction and then says "It now is not difficult to check which of the 10 types not investigated above can occur", and performs this check in one case. At the end of Hering 1985 there is a citation to a "to appear" paper by Hering (with a title — this is the final item in the bibliography below), and the assertion that that paper "contains the corresponding investigation for the known sporadic simple groups". But I checked Mathscinet, and could not find any paper of Hering's with the stated title, or even any paper of Hering's on this topic since his 1985 paper. Of course, the paper may still be "to appear", but perhaps after 31 years the odds are not high.
After digging through all of the above and getting increasingly frustrated, I was quite happy when a friendly group theorist pointed me to Liebeck 1987, which is the only reference I've seen which purports to prove the classification of affine 2-transitive groups without merely pointing the reader to one or more papers by Hering. (Minor issue: one must add the condition $a>1$ to case A2 of the result stated in Liebeck 1987, in order to make the result have content.) I am baffled by Liebeck's assertion that "The 2-transitive affine groups have been determined by Hering in [Hering 1985]", since he almost surely would have looked at the paper and noticed that it did not contain the result; but for whatever reason it seems taboo in the group theory community to admit that Hering 1985 (or even Hering 1974) does not even claim to prove this result.
But I also cannot follow the very first step of the proof in Liebeck 1987. It begins by letting $G_0$ be the stabilizer of the zero vector, so that $G_0\le\text{GL}_r(p)$, and choosing $a$ to be minimal so that $G_0\le\Gamma\text{L}_a(p^{r/a})$. Write $q:=p^{r/a}$. Cases with $a=1$ are included in the statement of the result, so one may assume $a>1$. The statement of the result also includes cases where $G_0$ normalizes $\text{SL}_a(q)$, so one may assume that this does not hold either. Liebeck then asserts that the main theorem of Aschbacher 1984 implies that one of these holds:
- $L\le G_0/Z(G_0)\le \text{Aut}(L)$ for some nonabelian simple group $L$ which isn't one of a few specific groups
- $G_0$ normalizes $\text{Sp}_a(q)$
- $G_0$ is contained in the normalizer in $\Gamma\text{L}(\text{F}_q^a)$ of some $\ell$-group $R$, where $\ell$ is a prime.
Aschbacher 1984 exclusively addresses groups between a simple group and its automorphism group, so it seems it can only be relevant to ruling out some of the groups stated not to occur in case 1. Does anyone see how to justify Liebeck's assertion? Or alternately, does anyone know any other reference containing a proof of the classification of nonsolvable affine 2-transitive groups?
Final remark: the above comments subsume all comments and answers to mathoverflow.32351.
================================================================
Bibliography:
Maillet 1895: Sur les isomorphes holoédriques et transitifs des groupes symétriques ou alternés, J. Math. Pures Appl. (5) 1 (1895), 5-34.
Huppert 1957: Zweifach transitive, auflösbare Permutationsgruppen, Math. Z. 68 (1957), 126-150.
Hering 1973: On linear groups which contain an irreducible subgroup of prime order, Proc. Int. Conf. Proj. Planes, Washington State Univ. Press, Pullman, 1973, pp. 99-105.
Hering 1974: Transitive groups and linear groups which contain irreducible subgroups of prime order, Geometriae Dedicata 2 (1974), 425-460.
O'Nan 1975: Normal structure of the one-point stabilizer of a doubly transitive permutation group I, II: Trans. Amer. Math. Soc. 214 (1975), 1-74.
Curtis-Kantor-Seitz 1976: The 2-transitive permutation representations of the finite Chevalley groups, Trans. Amer. Math. Soc. 218 (1976), 1-59. (For corrigenda, see the paper's Math.Review.)
Cameron 1981: Finite permutation groups and finite simple groups, Bull. London Math. Soc. 13 (1981), 1-22.
Huppert-Blackburn 1982: Finite Groups III. Springer-Verlag, Berlin, 1982.
Aschbacher 1984: On the maximal subgroups of the finite classical groups, Invent. Math. 76 (1984), 469-514.
Kantor 1985: Homogeneous designs and geometric lattices, J. Comb. Theory Ser. A (1985) 38, 66-74.
Hering 1985: Transitive groups and linear groups which contain irreducible subgroups of prime order, II: J. Algebra 93 (1985), 151-164.
Liebeck 1987: The affine permutation groups of rank three, Proc. London Math. Soc. (3) 54 (1987), 477-516.
Abhyankar 1992: Galois theory on the line in nonzero characteristic, Bull. Amer. Math. Soc. 27 (1992), 68-133.
Praeger-Soicher 1997: Low Rank Representations and Graphs for Sporadic Groups. Cambridge Univ. Press, Cambridge, 1997.
Dixon-Mortimer 1996: Permutation Groups. Springer-Verlag, New York, 1996.
Cameron 1999: Permutation Groups. Cambridge Univ. Press, Cambridge, 1999.
Hering future(?): On representations of Chevalley groups and translations planes, to appear (as of 1985).
Best Answer
The theorem of Aschbacher that is cited by Liebeck, looks at the maximal subgroups $G_0$ of a group $G$ satisfies $SL(a,p^{r/a})\leqslant G \leqslant \Gamma L(a,p^{(r/a})$. For the application of Liebeck, you take $G=\Gamma L(a,p^{r/a})$. Aschbacher gives 9 classes of such subgroups. They usually preserve some sort of geometric structure on the vector space and so you can easily see that a maximal subgroup of that form is not transitive on the set of nonzero vectors of $V$. This leaves the three classes stated.