Let $S_{G:H}$ be the set of permutations of $G:H$. Consider $\phi:(G:N)\rightarrow S_{G:H}$ that sends $xN$ to $T_x$ (Where $T_x$ is the permutation of $G:H$ that sends $aH$ to $xaH$ ). It is easy to verify that $\phi$ is an injective group homomorphism. Thus, $G:N$ is isomorphic to a subgrop of $S_{G:H}$. Using Lagrange's theorem, we deduce that $|G:N|$ divides $|S_{G:H}|=n!$
Kernel is the set of elements which map to identity under $\phi$
$Ker(\phi)=\{x\in\mathbb{R}^*: \phi(x)=1\}$
$Ker(\phi)=\{x\in\mathbb{R}^*: |x|=1\}$
Now which elements in $\mathbb{R}^*$ satisfy this?
Edit (Details):
You have $Ker(\phi)=\{1,-1\}$
Definition of Cosets: If $G$ is a group and $H$ is a subgroup of $G$, then
(i) a left coset of H in G is defined as
$gH=\{gh:h\in H\}$ where $g$ is a fixed element of $G$
i.e. if $H=\{h_1,h_2,\ldots,h_k\}$, then $gH=\{g\cdot h_1,g\cdot h_2,\ldots,g\cdot h_k\}$. Same is the case when $H$ is not finite.
(ii) Similarly, a right coset of H in G is defined as
$Hg=\{hg:h\in H\}$ where $g$ is a fixed element of $G$
Note: In your case, the left and right cosets are equal (Since $\mathbb{R}^*$ is abelian)
You can see it for the finite case below.
Suppose $H=\{h_1,h_2,\ldots,h_k\}$
Let $g\in G$ be arbitrary, then
$gH=\{g\cdot h_1,g\cdot h_2,\ldots,g\cdot h_k\}=\{h_1\cdot g,h_2\cdot g,\ldots,h_k\cdot g\}=Hg$
Now coming back to your question, let $r\in\mathbb{R}^*$
$$r\cdot Ker(\phi)=Ker(\phi)\cdot r=\{-r,r\}$$
So, the set of left cosets of $Ker(\phi)$ in $\mathbb{R}^*$ is $S=\{\{-r,r\}:r\in \mathbb{R^*}\}$
NOTE: This is also the set of right cosets of $Ker(\phi)$ in $\mathbb{R}^*$
Best Answer
Consider the action of $g$ on $g'H$. Then: $g$ acts trivially on $g'H$ for every $g' \in G$ $\leftrightarrow $ $gg'H=g'H$ for every $g'$. Taking $g'=$identity element, you see $g \in H$. But unless $H$ is normal, $h \in H$: $hg'H \ne g'H$ does not hold for all $h \in H$ and $g'\in G$. For example take $H=\{\begin{pmatrix} 1 & n \\ 0 & 1 \end{pmatrix},n \in \Bbb N\}$, $h=\begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$,$g'=\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$. Then $g'^{-1}hg'Id_n=\begin{pmatrix} -5 & -8 \\ 9/2 & 7 \end{pmatrix} \notin H$.