I think the Wikipedia article on commutators explains the definitions and properties well. If I can clarify anything said there, let me know. It's important to note that abelianization is a covariant functor, so in particular a homomorphism of groups induces one of the abelianizations.
This is a useful trick, not only because in applying $\pi_1$ you will often produce free products that seem hard to compare with one another. For example, $\mathbf Z * \mathbf Z$ contains free subgroups of all finite ranks, so it isn't clear (at least to me) that we can use van Kampen to show that the wedge of two circles and the wedge of three circles have non-isomorphic $\pi_1$s. But abelianizing makes this clear.
Back to the example. I claim that for a retract $M_h' \to C$ to exist, $i_*^\text{ab}\colon \pi_1(C) \to \pi_1(M_h')^\text{ab}$ would have to be injective (It does not always happen that the abelianization of an injection is injective; what's special here?). Now, $M_h'$ is the surface of genus $h$ with a hole in it, and we can represent that as
So the map $i_*$ sends a generator for $\pi_1(C)$ to a commutator $[a_1, b_1] \cdots [a_g, b_g]$. Why does this contradict the injectivity of $i_*^\text{ab}$?
For the second part, you'll need to give a map. Here's one idea: in the above picture, ignoring the $C$, try to retract onto $a_1$. It might help to redraw this as a square with sides $a_1, b_1, a_1$, and a bunch of stuff coming off of the fourth side. The square looks a lot like the torus, and you know how to retract the torus onto a meridian (?). Note that you really can crush a bunch of stuff on the polygon together as long as you respect the identifications.
By naturality, it suffices to prove the same statement for
$f_* : H_{n-1}(\mathbb{R}^n - \left\{0\right\}) \to H_{n-1} ( \mathbb{R}^n - \left\{0\right\})$ where $f$ can be assumed to be a reflection as you have noted.
Let $\overline{f}: S^{n-1} \to S^{n-1}$ denote the map which is restriction of $f$ to $S^{n-1}$. Let $i : S^{n-1} \to \mathbb{R}^n - \left\{0 \right\}$.
Then we have $f \circ i = i \circ \overline{f} $
Hence, $f_* \circ i_* = i_* \circ \overline{f}_*$ in homology.
Note that since $i_*$ is an isomoprhism (as $S^{n-1}$ is a deformation retraction of $\mathbb{R}^n - \left\{0\right\}$) and all the homology groups are isomorphic to $\mathbb{Z}$, $i_*$ must send the generator ($1$) to another generator i.e $(\pm 1$).
So, it is enough to prove the statement for the map $\overline{f}_* : H_{n-1}(S^{n-1}) \to H_{n-1}(S^{n-1}) $, where $\overline{f}$ is a reflection of $S^{n-1}$. Hence $\deg( \overline{f})$ is $-1$ and the claim follows.
Best Answer
Here is a restatement: $f$ induces a map of the pair $(R^n, R^n-{0})$ to itself. The $n$th homology of this pair is isomorphic to Z. The induced map on this homology group is $\pm 1$ depending on the sign of the determinant of $f$.