Not every left transversal is also a right transversal. The Group Properties Wiki has a list of subgroup properties that are stronger than "having [at least one] left transversal that is also a right transversal"; among these is normality as William notes.
However the left coset representatives' multiplicative inverses form a right transversal, because
$$\begin{array}{c l}xH=yH & \iff y^{-1}xH=H \\ & \iff y^{-1}x\in H \\ & \iff (y^{-1}x)^{-1}=x^{-1}y\in H \\ & \iff Hx^{-1}y=H \\ & \iff Hx^{-1}=Hy^{-1}. \end{array}$$
The "$a$" in the definition is any element of $G$: .
So the left coset $\,aH\subseteq G\,$ is the set of all elements in the left coset $aH$, which for a given $\,a \in G\,$ and every element $h_i \in H$, is the set of all $ah_i$.
E.g. Take a small subgroup of $S_3$ : $\;H = \langle (12)\rangle = \{id, (12)\} \leq S_3.\,$ There are three left (respectively right) cosets of $\,H$ in $\,S_3$. One coset is $\,H\,$ itself. The other cosets are $\,(13)H = (123)H\,$ and $\,(23)H = (132)H$.
You'll see that for any subgroup $\,H \leq G$, every element of $\,G\,$ will belong to one and only one left (respectively right) coset of $\,H\,$ in $\,G.\,$ And the union of all left cosets of $H$ in $G$ (respectively the union of all right cosets of $H$ in $G$) is $G$. That is, the left (respectively right) cosets of $H$ in $G$ partition $G$.
You'll can find a nice definition of "coset" and some examples here, as well.
Best Answer
Hint: If $g \in G$ and $H < G$ then $gh = (ghg^{-1})g$ for all $h \in H$.