For each $n\in \Bbb{Z}^+$, $n\geq 3$ let $D_{2n}$ be the set of symmetries of a regular $n$-gon, where a symmetry is any rigid motion of the $n$-gon which can be effected by taking a copy of the $n$-gon, moving this copy in any fashion in $3$-space and then placing the copy back on the original $n$-gon so it exactly covers it. More precisely, we can describe the symmetries by first choosing a labelling of the $n$ vertices
Then each symmetry $s$ can be described uniquely by the corresponding permutation $\sigma$ of $\{1,2,3,…,n\}$ where if the symmetry $s$ puts vertex $i$ in the place where vertex $j$ was originally, then $\sigma$ is the permutation sending $i$ to $j$. For instance, if $s$ is a rotation of $2\pi /n$ radians clockwise about the center of the $n$-gon, then $\sigma$ is the permutation sending $i$ to $i+1$, $1\leq i \leq n – 1$, and $\sigma(n) = 1$. Now make $D_{2n}$ into a group by defining $st$ for $s, t \in D_{2n}$ to be the symmetry obtained by first applying $t$ then $s$ to the $n$-gon (note that we are viewing symmetries as functions on the $n$-gon, so $st$ is just function composition – read as usual from right to left). If $s,t$ effect the permutations $\sigma$, $\tau$, respectively on the vertices, then $st$ effects $\sigma \circ \tau$. The binary operation on $D_{2n}$ is associative since composition of functions is associative. The identity of $D_{2n}$ is the identity symmetry (which leaves all vertices fixed), denoted by $1$, and the inverse of $s \in D_{2n}$ is the symmetry which reverses all rigid motions of $s$ (so if $s$ effects permutation $\sigma$ on the vertices, $s^{-1}$ effects $\sigma^{-1}$)
For each symmetry $s$, $\exists !\sigma \in S_n$ such that $\sigma$ describe $s$. Que: Can we define symmetry only based on permutation of $\{1,…,n\}$? Of course, we need to put more constraint on permutation to make it symmetry. Author said “we are viewing symmetry as function on the $n$-gon” but used permutation of $J_n$ to view symmetry. Former domain of function is $n$-gon and latter domain of function is $\{1,…,n\}$. So both functions are different. Assuming we view symmetry as permutation. Then $D_{2n}$ is set of all permutation with certain property. How to rigorously show $\sigma^{-1}\in D_{2n}$?
Here is mathematical definition of word “symmetry”. So $D_{2n}=\{T:\Bbb{R}^2\to \Bbb{R}^2\mid T(P)=P \text{ and } T \text{ is isometry}\}$, where $P$ denotes $n$-gon. We claim $D_{2n}$ is group under composition $\circ$. Let $T,U\in D_{2n}$. Then $T\circ U (P)=T(U(P))=T(P)=P$. In real analysis, we proved composition of two isometry function is an isometry. So $T\circ U$ is isometry. Thus $T\circ U\in D_{2n}$. $(1)$ Composition is associative. $(2)$ Clearly identity map $I$ on $\Bbb{R}^2$ is in $D_{2n}$. Thus $\exists I\in D_{2n}$ such that $T\circ I=I\circ T=T$, for all $T\in D_{2n}$. $(3)$ Let $T\in D_{2n}$. By definition of isometry, $T$ is bijective or invertible. Then $T^{-1}(P)=T^{-1}(T(P))=I(P)=P$. In real analysis, we proved $T^{-1}$ is isometry. So $T^{-1}\in D_{2n}$. Thus $\exists T^{-1}\in D_{2n}$ such that $T\circ T^{-1}=T^{-1}\circ T=I$. Hence $(D_{2n},\circ)$ is group.
Above abstract definition of $D_{2n}$ don’t help in proving order of $D_{2n}$ is $2n$. We have to go back to naive definition of symmetry. Define $r$ as rotation and $s$ as reflection.
Best Answer
So the question is whether one can describe precisely which permutations of $\{1,\ldots,n\}$ correspond to symmetries of the regular $n$-gon, if we label the vertices sequentially going around the $n$-gon, and use the permutation to describe the symmetry by describing where the vertices end up.
In a symmetry of the regular $n$-gon, once you know where $1$ ends up, say $i$, you have two possible choices for $2$: it is either in position $i+1$ or in position $i-1$. If it is in position $i+1$, then $3$ must be in position $i+2$, $4$ in position $i+3$, and so on (with addition taken "modulo $n$"). If $2$ is in position $i-1$, then $3$ must be in position $i-2$, $4$ must be in position $i-3$, and so on.
The first family corresponds to permutations $\sigma\in S_n$ with the property that for every $a\in \{1,\ldots,n\}$, $\sigma(a+1) = \sigma(a)+1$ (again, addition taken modulo $n$, so "$n+1$" means $1$). The second family corresponds to permutations in $\sigma\in S_n$ with the property that for every $a\in \{1,\ldots,n\}$, $\sigma(a+1)=\sigma(a)-1$ (where "$0$" means $n$). That means that knowing $\sigma(1)$ and $\sigma(2)$ will completely determine the permutation, and that $\sigma(2)$ must be either $\sigma(1)+1$ or $\sigma(1)-1$, corresponding to the two types of symmetries.
It is now straighforward to verify that the inverse of such a permutation is again of that type. If $\sigma$ is such that $\sigma(a+1)=\sigma(a)+1$ for every $a$, then let $b$ be such that $\sigma(b)=a+1$. Then $\sigma(b-1)=a$, so $\sigma^{-1}(a+1) = b = \sigma^{-1}(a)+1$. So $\sigma^{-1}$ also satisfies that $\sigma^{-1}(r+1)=\sigma^{-1}(r)+1$ for all $r$.
If $\sigma$ is such that $\sigma(a+1)=\sigma(a)-1$ for every $a$, then if $\sigma(b)=a+1$, we have $\sigma(b+1)=a$; so $\sigma^{-1}(a+1) = b = \sigma^{-1}(a)-1$; thus $\sigma^{-1}$ satisfies that $\sigma^{-1}(r+1) = \sigma^{-1}(r)-1$ for all $r$.
We can also verify that this set is closed under composition, though it takes a few cases. Say $\sigma_1$ and $\sigma_2$ satisfy $\sigma_i(a+1) = \sigma_i(a)+1$, and $\tau_1$ and $\tau_2$ satisfy $\tau_i(a+1) = \tau_i(a)-1$.
Note that if $\tau(a+1) = \tau(a)-1$ for all $a$, then $\tau(a)=\tau(a-1)-1$, so $\tau(a-1)=\tau(a)+1$. Similarly, if $\sigma(a+1)=\sigma(a)+1$ for all $a$, thenb $\sigma(a-1) = \sigma(a)-1$.
Then we have: $$\begin{align*} (\sigma_2\circ\sigma_1)(a+1) &= \sigma_2(\sigma_1(a)+1)=\sigma_2(\sigma_1(a)) + 1 = (\sigma_2\circ\sigma_1)(a)+1\\ (\tau_2\circ\tau_1)(a+1) &= \tau_2(\tau_1(a)-1) = \tau_2(\tau_1(a))+1 = (\tau_2\circ\tau_1)(a)+1\\ (\tau_2\circ\sigma_1)(a+1) &= \tau_2(\sigma_1(a)+1) = \tau_2(\sigma_1(a))-1 = (\tau_2\circ\sigma_1)(a) - 1\\ (\sigma_2\circ\tau_1)(a+1) &= \sigma_2(\tau_1(a)-1) = \sigma_2(\tau_1(a))-1 = (\sigma_2\circ\tau_1)(a) - 1. \end{align*}$$ So in all four cases, the composition is again one of the two types of permutations described. This will also tell you that the set is closed under inverses, since the inverse of a permutation is always equal to a power of that permutation.