I changed this question so that the socks don't have labels. The labels weren't important.
Suppose we have an infinite number of identical socks $s$. Call this infinite set $S$. Take one of the socks $s$ out of the set $S$ and we still have the original set $S$, since the socks are all identical and there are infinitely many, plus the sock $s$ that we took out, so we have $S \cup \{s\}$. But this new set $S \cup \{s\}$ is really just the same as the old set $S$, since the socks are all the same, so we have $S \cup \{s\}=S$. Subtracting $S$ on both sides, we obtain $\{s\}=\emptyset$, a contradiction. So there can be no infinite sets.
Best Answer
Recall that sets ignore both order and repetition of elements, so $\{0,0\}=\{0\}$. So if all the socks are identical then $S$ is really just a singleton. But if we know that the socks are indiscernible from one another, but still different we can still talk about the infinite set of socks.
Once you took out a sock you don't have the same set. If $S$ is all the socks except $s$ then $S\cup\{s\}\neq S$. Now your arguments amounts to cardinality and to the fact that both $S$ and $S\cup\{s\}$ have the same cardinality, and my previously deleted answer applies again:
$\infty+1=\infty-1=\infty$. You can't cancel it out.
$$\Huge\text{Infinity is not a finite number.}$$
You can't apply the rules of finite arithmetics to infinite sets.