The guarantee that such a set can't exist is already given by the argument of Russell's paradox: its existence leads to a contradiction therefore it can't exist.
The problem with unrestricted comprehension was that it guaranteed the set does exist, which causes a problem because of the conflicting guarantees.
See Russell's Paradox :
Zermelo replaces NC [Naïve Comprehension principle] with the following axiom schema of Separation (or Aussonderungsaxiom):
$$∀A ∃B ∀x (x \in B \iff (x \in A \land \varphi)).$$
Again, to avoid circularity, $B$ cannot be free in $\varphi$. This demands that in order to gain entry into $B$, $x$ must be a member of an existing set $A$. As one might imagine, this requires a host of additional set-existence axioms, none of which would be required if NC had held up.
How does Separation avoid Russell's paradox? One might think at first that it doesn't. After all, if we let $A$ be $V$ – the whole universe of sets – and $\varphi$ be $x ∉ x$, a contradiction again appears to arise. But in this case, all the contradiction shows is that $V$ is not a set. All the contradiction shows is that “$V$” is an empty name (i.e., that it has no reference, that $V$ does not exist), since the ontology of Zermelo's system consists solely of sets.
Consider a set $z$, we can still form the set $R = \{ x \in z : x \notin x \}$;
This only implies that :
if $R \in z$, then $R \in R \ \text { iff } \ R \notin R$, which means (reductio ad absurdum) that $R \notin z$.
Thus :
there is no universal set : $\forall z \exists R(R \notin z)$.
It can be worth to note that the "non-existence" of the Russell's set $R$ can be proved by logic alone :
1) $\exists y \forall x(A(x,y) \iff \lnot A(x,x))$ --- assumed [a]
2) $\forall x(A(x,c) \iff \lnot A(x,x))$ --- with $c$ a new constant
3) $A(c,c) \iff \lnot A(c,c)$ --- by instantiation.
The last line gives us a contradiction, because : $\vdash A(c,c) \iff A(c,c)$; thus, we conclude with :
$\vdash \lnot \exists y \forall x(A(x,y) \iff \lnot A(x,x))$ --- (*)
discharging the assumption [a].
Now, if we apply (*) to the language of set theory with the binary predicate $\in$ in place of $A$, we get :
$\lnot \exists y \forall x((x \in y) \iff (x \notin x))$.
Best Answer
Axiom of regularity (or Foundation) rules out the case $S\in S$.
Otherwise, we would have $S\notin S$. Now does this imply $S\in S$?
The problem with Russel's paradox is it implicitly implies existence of universal set ("set of all sets").
If $A$ is a set and $S=\{x\in A:x\notin x\}$, $S\notin S$ implies either $S\in S$ or $S\notin A$.
In case of Russell's paradox, it is not possible to have $S\notin A$, because $S$ must belong to set of all sets, if that existed, and leads us to paradox $S \in S$. But with our construction such set cannot exist (exactly because existence of such set leads us to Russell's paradox). So paradox does not arise.