I've found various equivalent definitions of normal subgroup from this Wikipedia page. I've just finished proving their equivalence. Could you please verify if it is fine or contains logical mistakes?
Let $N$ be a subgroup of $G$. Then the following statements are equivalent.
a. For all $g,h \in G$: $gh \in N \iff hg \in N$.
b. For all $g \in G$: $gNg^{-1} \subseteq N$.
c. For all $g \in G$: $gNg^{-1} = N$.
d. The sets of left and right cosets of $N$ in $G$ coincide.
e. For all $x,y,g,h \in G$: $x \in gN$ and $y \in hN \implies xy \in (gh)N$.
f. For all $n \in N, g \in G$: $n^{-1} g^{-1} n g \in N$.
g. For all $n \in N, g \in G$: $g n g^{-1} \in N$.
h. $N = \bigcup_{n \in N} \operatorname{Cl}(n)$ where $\operatorname{Cl}(n) := \{gng^{-1} \mid g \in G\}$.
i. For all $g \in G$: $gN = Ng$.
j. There exists a group homomorphism whose domain is $G$ and kernel is $N$.
My attempt: To make it easier to follow, I put each part of the proof between two consecutive definitions.
For all $g,h \in G$: $gh \in N \iff hg \in N$.
Assume the contrary that there exist $n\in N$ and $g \in G$ such that $gng^{-1} \notin N$. Then $g^{-1} (gn) = n\notin N$ by (a). This is a contradiction.
For all $g \in G$: $gNg^{-1} \subseteq N$.
Substituting $g$ for $g^{-1}$ in $gNg^{-1} \subseteq N$, we get $g^{-1} N g\subseteq N$. Substituting $N$ for $g^{-1} N g$ in $gNg^{-1} \subseteq N$ , we get $N \subseteq g^{-1} N g$. As a result, $g^{-1} N g = N$. Substituting $g$ for $g^{-1}$ in $g^{-1} N g = N$, we get the desired result.
For all $g \in G$: $gNg^{-1} = N$.
It follows from (c) that $gN= Ng$. The result then follows.
The sets of left and right cosets of $N$ in $G$ coincide.
Because of (d) and the fact that $h \in hN \cap Nh$, we have $hN=Nh$. It follows from $x \in gN$ and $y \in hN$ that $x = gn_1$ and $y = hn_2$ for some $n_1,n_2 \in N$. Then $xy = gn_1 hn_2$. Because $Nh = hN$, $n_1 h = h n_3$ for some $n_3 \in N$. Then $xy = g h n_3 n_2 = (gh) (n_3 n_2) \in (gh) N$.
For all $x,y,g,h \in G$: $x \in gN$ and $y \in hN \implies xy \in (gh)N$.
We have $n^{-1} g^{-1} n \in n^{-1} g^{-1} N$ and $g \in gN$. Then by (e), we have $n^{-1} g^{-1} n g \in (n^{-1} g^{-1} g)N = n^{-1} N = N$.
For all $n \in N, g \in G$: $n^{-1} g^{-1} n g \in N$.
Substituting $g$ for $g^{-1}$ in $n^{-1} g^{-1} n g \in N$, we get $n^{-1} g n g^{-1} \in N$. Because $n^{-1} \in N$, we have $g n g^{-1} \in N$.
For all $n \in N, g \in G$: $g n g^{-1} \in N$.
It follows from (g) that $\operatorname{Cl}(n) \subseteq N$ for all $n \in N$. Then $\bigcup_{n \in N} \operatorname{Cl}(n) \subseteq N$. On the other hand, $n \in \operatorname{Cl}(n)$ and thus $N \subseteq \bigcup_{n \in N} \operatorname{Cl}(n)$. The result the follows.
$N = \bigcup_{n \in N} \operatorname{Cl}(n)$ where $\operatorname{Cl}(n) := \{gng^{-1} \mid g \in G\}$.
It follows from (h) that $\operatorname{Cl}(n) \subseteq N$ for all $n \in N$. As a result, for all $g \in G, n \in N$, we have $g n g^{-1} = n'$ for some $n' \in N$. Hence $gn=n'g$ for some $n' \in N$. Thus $gN \subseteq Ng$. By symmetry, we also have $Ng \subseteq gN$. The result then follows.
For all $g \in G$: $gN = Ng$.
Let $G/N := \{gN \mid g \in G\}$. We define a binary operation $G/N \times G/N \to G/N$ by $(gN) (hN) \mapsto (gh)N$. Let's prove that it's well-defined, i.e. $gN = aN$ and $hN = bN$ implies $(gh) N = (ab) N$.
We have $gN= aN$ implies $g=an_1$ for some $n_1 \in N$. Similarly, $h=bn_2$ for some $n_2 \in N$. Then $gh = an_1bn_2$. It follows from (i) that $Nb=bN$ and thus $n_1b=bn_3$ for some $n_3 \in N$. Hence $gh =abn_3n_2$. Because $n_2,n_3 \in N$, we have $n_3n_2 \in N$ and thus $(n_3n_2)N =N$. As a result, $(gh)N = (abn_3n_2)N = (ab)(n_3n_2)N = (ab)N$.
It's then straightforward to verify that $G/N$ together with above operation is group. Now we define a map $\phi: G \to G/N, g \mapsto gN$. It's easy to verify that $\phi$ is in fact a group homomorphism such that $\operatorname{ker} \phi = \{g \in G \mid gN = 1N =N\} =N$.
There exists a group homomorphism whose domain is $G$ and kernel is $N$.
Let $\phi: G \to K$ be such a group homomorphism that $\operatorname{ker} \phi = N$. If $gh \in N$ then $\phi (gh) = \phi(g) \phi (h) = 1$. This means $\phi(g) = (\phi(h))^{-1}$. As a result, $\phi (hg) = \phi(h) \phi (g) = \phi(h) (\phi(h))^{-1} = 1$ and thus $hg \in N$. By symmetry, we have $hg \in N \implies gh \in N$. This completes the proof.
For all $g,h \in G$: $gh \in N \iff hg \in N$.
Best Answer
Here is @Arturo Magidin's comment that answers my question. I post it here to remove this question from unanswered list. All credits are given to @Arturo Magidin.