Here are two motivating problems. I will begin with $S_4$.
Problem 1. Why is there no proper non-trivial normal subgroup of order 2 in $S_4$?
Using the class equation, we know that
$$|S_4| = |Z(S_4)| + \sum_{i = 1}^N [S_4 : C_{S_4}(g_i)]$$ where $N$ is the number of non-singular conjugacy classes in $S_4$ (the singular conjugacy classes are counted in $|Z(S_4)|$) and $g_i$ are representative elements for each of the $N$ conjugacy classes. There's a trick we can use to calculate the conjugacy classes of $S_4$: the fact that the conjugacy classes of $S_4$ correspond to the "shape" of elements when each element is written in cycle notation. These are representative elements of the conjugacy classes.
$$E = \{(12), (123), (1234), (12)(34)\}$$
And these are how the orders of the conjugacy classes are calculated for all $e \in E$. Recall that $[S_4 : C_{S_4}(g_i)]$ is equal to the size of the conjugacy class that contains $g_i$.
$[S_4 : C_{S_4}((12))] = {4\choose 2} = 6$
$[S_4 : C_{S_4}((123))] = {4\choose3}2 = 8$
$[S_4 : C_{S_4}((1234))] = 4!/\langle \text{symmetry of 4-cycle} \rangle = 24/4 = 6$
$[S_4 : C_{S_4}((12)(34))] = {4\choose 2}/\langle \text{symmetry from the fact that disjoint cycles commute} \rangle = 6/2 = 3$
So the class equation is expanded as thus:
$$|S_4| = 1 + 6 + 8 + 6 + 3.$$
Because a normal subgroup is any subgroup $H$ such that $gHg^{-1} = H$ for any element $g \in S_4$, any normal group can only be made from whole conjugacy classes, not a part, which means they can only be addition subsets of the class equation. In conjunction with Lagrange's theorem (the fact that the order of the subgroups of $S_4$ must divide $24$), $2$ is not in the intersection of possible sums of the class equation with possible divisors of 24.
Problem 2. Why is there no proper nontrivial normal subgroup of order 6 in $A_4$.
The problem is that I can't use the fact that elements of the same equivalence class have the same shape when written in cycle notation, because $A_4$ only has even permutations. So it could be possible that for $x, y \in S_4$, $x = g y g^{-1}$ only for an odd permutation $g$, which would make them conjugate in $S_4$ but non-conjugate in $A_4$. How would I count the conjugacy classes in $A_4$?
Best Answer
Well, actually, we can use that fact. We just can't stop there. We can find the sets of permutations with the same cycle shapes, and then we'll have to check to see which of these split further into pairs of conjugacy classes.
$1+1+1+1$: The identity. A single conjugacy class of one element, which obviously doesn't split.
$2+2$: Pairs of disjoint transpositions. There are three of these, and they can't split. Why? Because any split would have to be into two subsets of equal size. Conjugation by a transposition would be a bijection between the two classes.
$3+1$: $3$-cycles. There are eight of these, and this set does split into two conjugacy classes of size $4$ each. As it turns out, a $3$-cycle and its inverse aren't conjugate in $A_4$; any even permutation which leaves the fixed element alone commutes with the cycle.
Alternately, we can prove this splits with an orbit-stabilizer count. $A_4$ acts on itself by conjugation, and the orbits are the conjugacy classes. The stabilizer of a $3$-cycle consists of all permutations that fix the fourth element not in that cycle, which are the powers of that $3$-cycle. That's a subgroup of order $3$, leaving room for four elements in the orbit. There's no way to get an eight-element orbit, because $\frac{12}{8}$ isn't an integer; the size of any conjugacy class must always divide the order of the group.
The other shapes are all odd permutations.
So then, the class equation is $$|A_4|=1+3+4+4$$ No way to get $6$ out of that. Every subgroup contains the identity, so the only possible orders of normal subgroups are $1$, $1+3=4$, and $1+3+4+4=12$. All three of those possibilities are in fact normal subgroups in this case.