Can you kindly provide an example of a subgroup that is not normal? I have been told many times that, for coset multiplication to be defined, the subgroup must be normal. I have seen the proof and examples of quotient group multiplications. Now, I am trying to find out where the process will break down if the subgroup is not normal.
Suppose that $x \in a_1H = a_2H, y \in b_1H = b_2H$, where $H$ is not normal.
So $x = a_1h_1 = a_2h_2$ for some $h_1, h_2 \in H$, and $y = b_1h_1^* = b_2h_2^*$ for some $h_1, h_2 \in H$.
$xy = (a_1b_1)(h_1h_1^*) = (a_2b_2)(h_2h_2^*)$. However, both $h_1h_1^*, h_2h_2^*$ are still elements of $H$.
So I don't see how $a_1b_1H \neq a_2b_2H$.
Best Answer
Note that in your proof, $xy = (a_1h_1)(b_1h_1^*)$. When you write that $xy = (a_1b_1)(h_1h_1^*)$ is assuming the group in question is abelian. But this assumption is unwarranted, because when commutativity does not hold, we cannot assume $xy = (a_1h_1)(b_1h_1^*)=(a_1b_1)(h_1h_1^*)$.
Indeed, if a group is abelian, then every one of its subgroups are normal, as you've shown to be the case, but this doesn't hold in nonabelian groups.
Consider, for example, the group $G = S_3$ and the subgroup $H\lt G$, $H = \{id, (12)\}$. This group is not normal in $S_3$. Consider other non-abelian groups, as well, to find many other counterexamples.