One of the key concepts behind quotient objects in an algebraic structure is the notion of congruence. But first, behind this there's an even more primitive notion, of equivalence. Now an equivalence relation on a set is really nothing more than forming a partition of "clumps" (subsets) of our original set. A very "useful" such partition is the "pre-image" partition of the domain of a function $f: A \to B$, in which our equivalence relation (on $A$) is formally given by:
$a_1 \sim a_2 \iff f(a_1) = f(a_2)$ so that $[a] = f^{-1}(f(a))$.
This lumps all pre-images of an element in $B$ together in the same equivalence class.
Equivalence is like a "relaxation" of equality, which nevertheless enjoys the properties of reflexiveness, symmetry, and transitivity that we use almost without thinking with the equals sign.
With a congruence, we have an equivalence relation that "respects" the algebraic operations of our structure. For example, an ADDITIVE congruence would be an equivalence relation on a set $S$ with an addition, +, such that:
if $[a] = [a']$ and $[b] = [b']$, then we also have: $[a + b] = [a' + b']$.
Let's see how this plays out in the context of groups. Now, any group $G$ has a "special" element, $e$, its group identity. So if we have a group congruence, perhaps the equivalence class of the identity is a special one, as well. Since we have a congruence, we know that:
$[g] = [e]$, and $[g'] = [e]$ together mean that $[gg'] = [ee] = [e]$, which means that if $g,g' \in [e]$, so is $gg'$, the equivalence (congruence) class of the identity is closed under the group multiplication. Now let's look at inverses:
Suppose that $g \in [e]$, that is, $[g] = [e]$. We would like to know what $[g^{-1}]$ is. Since we don't know, yet, let's just call it $[b]$. Since we have a congruence, we know that:
$[gg^{-1}] = [eb] = [b]$. But the (far) LHS is just $[e] = [gg^{-1}]$, so putting the two facts together, we have:
$[g^{-1}] = [b] = [e]$, which shows if $g \in [e]$, so is $g^{-1}$. Hence the equivalence class of $e$ is indeed special, it's a SUBGROUP of $G$, which is an unexpected bonus. So now we know there's more "structure" to our "clumps" of $G$ (the equivalence/congruence classes) than it might appear, at first.
Naturally, we might ask: what does a "typical" congruence class "look like"? I claim that:
$[g] = \{gx: x \in [e]\} = L_g$ (you probably know $L_g$ better as a left coset of $[e]$).
To show that $L_g \subseteq [g]$, we'll demonstrate that for each and every $x \in [e]$, that $[gx] = [g]$. From $x \in [e]$, we have $[x] = [e]$, and certainly $[g] = [g]$, so because we have a congruence:
$[gx] = [ge] = [g]$.
On the other hand, suppose that $y \in [g]$, so that $[y] = [g]$. Now $y = g(g^{-1}y)$, so we wish to show that $x = g^{-1}y \in [e]$. Again, since we have a congruence, we have from $[y] = [g]$ and $[g^{-1}] = [g^{-1}]$ that:
$[x] = [g^{-1}y] = [g^{-1}g] = [e]$, which shows $[g] \subseteq L_g$.
Now normally, the idea of congruence modulo a subgroup is introduced, for quotient groups. This is the same idea as $[e]$, which we already saw was in fact a subgroup, which we might as well call $H$. As we saw, the equivalence classes of a congruence, are of the form $gH = \{gh: h \in H\}$. The only "missing link" at this point, is showing that the equivalence class of the identity, $[e] = H$ is a special KIND of subgroup, called a NORMAL subgroup. This entails showing that $[g] = gH = Hg$, which is clear from the fact that we have a congruence (we show that $[xg] = [g]$ for $x \in H$, the same way we showed $[gx] = [g]$).
Let's go "one step further" to give the equivalence relation for a congruence as it is usually defined: for a normal subgroup $H$ of $G$, the equivalence relation given by: $x \sim y \iff x^{-1}y \in H$ is, in fact, a congruence. Note that this is really the same as saying $x \sim y \iff xH = yH$. We show this is a congruence, the usual way:
Let $[x] = [x']$ (that is, $xH = x'H$), and $[y] = [y']$ (so $yH = y'H$). We need to demonstrate $[xy] = [x'y']$, that is: $(xy)^{-1}x'y' \in H$. Now:
$(xy)^{-1}x'y' = y^{-1}x^{-1}x'y'$. Since $[x] = [x']$, we know $x^{-1}x' \in H$. Thus we need to show $y^{-1}Hy' = H$, or equivalently, $Hy' = yH$. Here is where the normality of $H$ is crucial:
BECAUSE $H$ IS NORMAL, $Hy' = y'H$, and since $[y] = [y']$, we have $Hy' = y'H = yH$. Working backwards, we then know that $y^{-1}Hy' = H$, and since $x^{-1}x' \in H$, so is $y^{-1}x^{-1}x'y'$, and we have a congruence.
The idea of a quotient group, or quotient ring, does seem unnatural, at first. The key idea is that it allows us to "clump" together "equivalent members" of a group, in such a way that we get a smaller group. The same idea applies to congruences of rings, which have to "respect" BOTH addition and multiplication, and the type of "thing" the congruence class of the identity (zero) becomes, is called an IDEAL.
Best Answer
When $G$ is nonabelian, its elements of finite order are not a subgroup in general (if $g$ and $h$ have finite order, $gh$ could have infinite order). When $G$ is abelian and $H$ is its subgroup of elements of finite order, the fact that $G/H$ has no nontrivial elements of finite order requires an argument. To appreciate this, if a group $G$ has a center $Z$, which is always a normal subgroup, the quotient group $G/Z$ might have a nontrivial center. Passing from $G$ to $G/Z$ does not have to "kill the center" in $G/Z$ because the center of $G/Z$ is represented by elements of $G$ that commute with everything in $G$ "up to multiplication by elements of $Z$", so when $Z$ is killed off, you can be left with nontrivial elements of $G/Z$ that now commute with everything in $G/Z$.
Example: $G = D_4 = \langle r,s\rangle$ where $r^4 = 1$, $s^2 = 1$, and $srs^{-1} = r^{-1}$. The center of $G$ is $Z = \{1,r^2\}$ and $G/Z$ is a group of order 4, which is abelian (all groups of order 4 are abelian) and therefore $G/Z$ doesn't have a trivial center: the whole quotient group $G/Z$ is its center. What is going on here is that when you pass from $G$ to $G/Z$, two elements $g$ and $h$ in $G$ that may not commute in $G$ could commute in $G/Z$ because in $G$ they only commute "up to an element in the center": $hg = zgh$ for some nontrivial $z$ in $Z$. Then $g$ and $h$ do not commute in $G$ but their images in $G/Z$ do commute in $G/Z$.