The integers $m_1$ and $m_2$ serve to bound $nx$ between two integers. The set $$=\{-m_2+1,-m_2+2,\ldots,m_1\}$$ is a finite set of integers, so we can choose the smallest member $m$ of this set such that $nx<m$.
If we knew that $nx$ was positive, we wouldn’t need $m_2$: we could just choose the smallest positive integer $m$ such that $nx<m$, since every non-empty set of positive integers has a least element. In fact, we can use that well-ordering principle directly, once we have $m_1$ and $m_2$. Let
$$M=\{m\in\Bbb Z^+:m-m_2>nx\}\;.$$
Then $M\ne\varnothing$, since $m_1+m_2\in M$, so $M$ has a least element, say $k$. Let $m=k-m_2$. Then $m>nx$. However, $k-1\notin M$, so $m-1=k-1-m_2\not>nx$, i.e., $m-1\le nx$. But note that I needed both $m_1$ and $m_2$ to carry out this argument: $m_1$ is needed to ensure that there is at least one integer that’s big enough to exceed $nx$, and $m_2$ is needed to ensure that not every integer is big enough.
Here is my interpretation of how Rudin argued the first point in your question (to contradict the case that $y^{n} < x$).
The idea is to find a $y^*$ that would lie in the "gap" of numbers created if $y$ were indeed less than the "true $\sqrt[n]{x}$". (Here $y$ refers to $y=\sup E$ where $E=\{t:t^n < x, t\in\mathbb{R^+}\}$ as defined by Rudin.)
In other words, we are trying to construct such a $y^*$ that has the two properties $(y^*)^n < x$ and $y^* > y$. The first property says that $y^* \in E$, the set which $y$ bounds above. The second property contradicts the fact that $y$ is an upper bound of $E$, since $y^* \not\le y$ and $y^* \in E$.
To construct $y^*$ we need to find a suitable $h$ such that $y^*=y+h$.
Intuitively, one would like to let $h$ be a positive quantity less than the difference $\sqrt[n]{x} - y$. However, we have not yet shown the existence of $\sqrt[n]{x}$ so this complicates our approach.
One alternative is to look at the difference $x - y^n$, which is a valid expression at this point in the proof. Graphically, this approach can be thought of as picking the position of $y+h$ on the horizontal axis based on the function value $(y+h)^n$ on the vertical axis.
In other words, instead of specifying $h$ directly we will try to specify $h$ in terms of what $y+h$ maps to under exponentiation by $n$.
Hence, we look for an $h$ such that $(y+h)^n - y^n < x - y^n$.
(Again, think of comparing function values of the curve $f(t) = t^n$ on a graph. The left-hand side $(y+h)^n - y^n$ is some positive quantity smaller than the height of the vertical "gap" assumed to exist between $y^n$ and $x$ on the graph.)
(Since $f(t) = t^n$ is a strictly increasing function (for positive $t$), inequalities are preserved, so the hypothetical horizontal "gap" (in which we were originally interested) will correspond with a vertical "gap" after this transformation. Intuitively, (pretending $\sqrt[n]{x}$ is defined) we could write,
$$
\sqrt[n]{x} - y > 0 \iff \sqrt[n]{x} > y \iff f(\sqrt[n]{x}) > f(y) \iff \sqrt[n]{x}^n > y^n \iff x > y^n \iff x - y^n > 0
$$
with the following justifications for each of the double arrows (in order from left to right): (1) addition/subtraction by $y$, (2) $f$ is strictly increasing, (3) definition of $f$, (4) definition of $\sqrt[n]{x}$, (5) addition/subtraction by $y^n$.)
Using the observation Rudin made$^1$ we can inject another expression into the inequality to get:
$(y+h)^n - y^n < hn(y+h)^n < x - y^n$
(This "injection" is okay for now because we have technically not defined $h$ yet, but instead are still working backwards to specify $h$.)
The trouble with this inequality is that the middle expression contains $h$ inside a binomial term raised to the power of $n$, which makes it hard to algebraically isolate. I believe Rudin makes the convenient assumption at this point that $h$ is small — in other words $0 < h < 1$ — in order to complete his definition of $h$.
We can now inject another expression into the inequality to yield:
$(y+h)^n - y^n < hn(y+h)^n < hn(y+1)^n < x - y^n$
Isolating just the rightmost two expressions and rearranging gives the final inequality:
$h < \frac{x - y^n}{n(y+1)^n}$
End Note
- Rudin's observation:
$b^n - a^n = (b-a)(b^{n-1} + ab^{n-2} + ... + a^{n-2}b + a^{n-1})$
so
$b^n - a^n < (b-a)n(b^{n-1})$
in the case $0 < a < b$.
(Rudin let $b = y+h$ and $a=y$ in the proof.)
Best Answer
Suppose that there were two distinct real numbers $y_1,y_2>0$ such that $y_1^n=x$ and $y_2^n=x$.
Because $y_1\neq y_2$, without loss of generality we have that $y_1<y_2$ (in other words, if it were instead the case that $y_2<y_1$, just relabel them). But as is pointed out in Rudin's argument, this implies that $y_1^n<y_2^n$, and therefore $x<x$, which is a contradiction; therefore our assumption that there were two distinct real numbers whose $n$th power was $x$ was false. Therefore there can be at most one positive real number $y$ such that $y^n=x$.