Your idea is correct, but the moving $p$ is detail to be taken care of. I like to fix bad homotopies by messing with the square, since the space is too complicated. Imagine starting at the bottom-right corner of the square. Then going backwards along the bottom is going along the path $\sigma(t)$ of the point $p$. That is, $\sigma(t) = h(t,p)$. If we then go up, this goes along our path $\gamma$, and finally we go along the top, tracing $\sigma$, in the forward direction. So your square seems to be a "weird homotopy" from $\sigma^{-1} \ast \gamma \ast \sigma$ to the constant path, with the former path along three of the sides instead of just one. However, I claim that a clever transformation $I^2 \rightarrow I^2$ can turn this "weird homotopy" you defined into an honest homotopy from $\sigma^{-1} \ast \gamma \ast \sigma$ to the constant path.
EDIT: You must, of course, then figure out how to move from this to the fact that $\gamma$ is homotopic to the constant path. For this, you will either apply a nice theorem or attempt another fudge map for your square.
An important detail is that the fundamental group is built from loops that all start and end at a common base point. We know that some loops can be continuously deformed into other loops; these loops are called homotopically equivalent. As a special case, some loops can be continuously deformed into other loops while keeping the base point fixed in place.
The fundamental group consists of loops which are considered equivalent when they are homotopic in this special base-point-preserving sense. The fundamental group fails to be commutative when there are loops that are homotopically equivalent in the generic sense, and yet not homotopically equivalent in the base-preserving sense.
In pictures, below you can see the plane with two holes along with three loops a, b, c with a common base point.
Note that you can't continuously deform b into c unless you're allowed to move the base point. (This is our signal that the fundamental group doesn't commute, which we'll show in more detail.)
Loops b and c aren't homotopic in a way that preserves the base point. Therefore, b ≢ c in the fundamental group.
Even so, b and c are homotopic in the general sense. For example, one (non-base-point-preserving) homotopy from b to c involves sliding the base point around the lower hole:
A neat observation is that during this homotopy, the trajectory of the base point is recognizable as loop a!
Loops b and c are homotopic in the general sense. The trajectory of the base point under that homotopy is loop a.
From this, it follows that there's a base-preserving homotopy between loop c•a ("go around the hole, then perform c") and loop a•b ("perform b, then go around the hole"):
Loops c•a and a•b are equivalent in the fundamental group. In other words, loops b and c are conjugates:
$$ca \equiv ab$$ $$c \equiv a\cdot b \cdot a^{-1}$$
If the fundamental group were abelian, then this conjugacy would imply that b = c. Because b ≢ c, the fundamental group is nonabelian.
In general, when there's a homotopy between two loops b and c, let a be the trajectory of the base point under that homotopy. Then a is itself a loop with the same base point as b and c, and c= aba⁻¹ (b and c are conjugates). If there isn't a base-preserving homotopy, then a is nontrivial and b ≢ c, so the fundamental group isn't abelian.
Final comment: Note that from the beginning, the source of the problem (noncommutativity) is the presence of holes. We saw intuitively that we could deform b into c, but only by moving the base point— there's a hole in the way. Now we can see that property emerge algebraically:
Loops b and c are deformable into one another when they're conjugate c= aba⁻¹. The element a describes the trajectory of the base point during that deformation. So if they're deformable without moving the base point, this is equivalent to saying that conjugacy holds when a is trivial (topologically, a is the constant loop at the base point; algebraically, it's the multiplicative unit) in which case we have b = c. Otherwise, if we must move the base point in order to deform one loop into the other, this is equivalent to saying that conjugacy c= aba⁻¹ only holds for a nontrivial choice of loop a. A loop is nontrivial just when it encircles a hole (!), hence this is an algebraic way of saying that a hole (call it hole A) stands in the way of deforming b into c.
Best Answer
Taking the stereographic projection originating from one of the points in $S^m$, we have an homeomorphism between $S^n- S^m$ and $\mathbb{R}^n-\mathbb{R}^m$. The latter is homotopy equivalent to $\mathbb{R}^{n-m} \backslash \{0\},$ which in turn is homotopy equivalent to $S^{n-m-1}$. Hence, $S^n-S^m \stackrel{hmtp}{\simeq} S^{n-m-1}$.
It follows that if $n-m=2$, then the fundamental group is $\mathbb{Z}$. If $n-m=1$, we have two (path)-connected components both with trivial fundamental group. (This is clear from the start: $S^{n+1}-S^{n}$ is just two disks.) In all other cases, we are left with a connected set with trivial fundamental group.