What does $\sup(s_n)$ mean ?
In this context, I believe that they mean $\sup_{k\geq n}(s_k)$ so that the expression for the upper limit is $\lim_{n\rightarrow \infty} \sup_{k\geq n}(s_k)$. Note that you can have some numbers in your sequence larger than the upper limit. In fact, every number in the sequence can be larger than the upper limit, take the sequence $\{\frac{1}{n}\}$. Its upper limit is 0, but every number in the sequence is larger than 0. Another important property is that a sequence converges in $\mathbb{R}$ if and only if the upper and lower limits are in $\mathbb{R}$ and coincide.
Edit: The supremum of the set $\{\frac{1}{n}\}$ is 1, but the upper limit is not 1. To compute the upper limit, the idea is this. Take your sequence $\{s_n\}$ and form a new sequence, call it $\{t_n\}$ (they all may be infinite). Define $t_1$ to be the sup of $\{s_1,s_2,s_3\ldots\}$. Define $t_2$ to be the sup of $\{s_2,s_3,s_4\ldots\}$. $t_3$ to be the sup of $\{s_3,s_4,s_5\ldots\}$. and so on. Now check that $\{t_n\}$ is a decreasing sequence (you are throwing away potentially larger terms). Therefore it has a limit (maybe infinite). This limit is the upper limit.
Edit 2: Now for the lower limit. Start with the given sequence $\{s_n\}$. The lower limit is $\lim_{n\rightarrow \infty}\inf_{k\geq n}(s_k)$. Let's unravel this definition as we did for the upper limit. Define a new sequence of numbers $\{t_n\}$ (possibly including $-\infty$) as follows. Define $t_1$ to be the inf of $\{s_1,s_2,s_3\ldots\}$. Define $t_2$ to be the inf of $\{s_2,s_3,s_4\ldots\}$. $t_3$ to be the inf of $\{s_3,s_4,s_5\ldots\}$. and so on. Now check that $\{t_n\}$ is an increasing sequence, and therefore has a limit (possibly infinite). This limit is called the lower limit.
I'm going to drop the boundedness condition (it is a legitimate difference) and use the extended reals. Also, I'm doing it for $\limsup$ ($\liminf$ is analogous).
Let $\{a_n\}_{n\geq 0}$ be a sequence in $\mathbb{R}$.
Let $E$ be the set of sub-sequential limits as in $(1)$. I will show that it is closed.
Let $x$ be a limit point of $E$. Then there are $x_n\in E$ such that $x_n\to x$ as $n\to\infty$. Each $x_n$ is the limit of a subsequence $a^n_{k_j}$.
Suppose $x\in\mathbb{R}$. For all $\epsilon>0$ there is $N$ such that $\vert x-x_n\vert<\frac{\epsilon}{2}$ if $n>N$. Furthermore, there is some index $j_n$ such that $\vert x_n-a^n_{k_{j_n}}\vert<\frac{\epsilon}{2}$. We will choose the indices $j_n$ so that $k_{j_{n+1}}>k_{j_{n}}$ (they can be chosen inductively, starting with $j_1$). Now we have $a^n_{k_{j_n}}$, a subsequence of $a_k$.
Given $\epsilon>0$, we get $N$ (since $x_n$ is convergent) and if $n>N$ we have
$\vert x-a^n_{k_{j_n}}\vert\leq\vert x-x_n\vert+\vert x_n-a^n_{k_{j_n}}\vert<\frac{\epsilon}{2}+\frac{\epsilon}{2}=\epsilon$
Consequently, $x$ is a sub-sequential limit and $x\in E$.
For the cases that $x=\pm\infty$ you reword the same idea into bounding below or bounding above respectively.
So $E$ contains its limit points and is closed.
(1) is equivalent to (2)
Since $E$ is closed, $\sup E\in E$ and there is a subsequence with $\sup E$ as its limit.
Let $U=\sup E$. If there were $\epsilon>0$ such that for every $N>0$ there is $n>N$ with $a_n\geq U+\epsilon$, then there would be a subsequence with limit greater than $U$, but $U$ is the supremum of the sub-sequential limits. This proves (i).
Since there is a subsequence converging to $U$ we get (ii) by letting the $n$ be the index of an element in this subsequence.
So $\sup E$ satisfies the criteria of $U$ in (2).
(2) is equivalent to (3)
Note that $s_n$ is a weakly decreasing sequence. Since it is monotone, it has a limit (it's bounded below by $-\infty$ and we are in the extended reals).
Let $U$ satisfy the criteria in (2). By (ii), we can see that for every $\epsilon>0$, $s_m>U-\epsilon$. So $\lim s_n\geq U$.
By (i) we can see that for every $\epsilon>0$ there is $N>0$ such that $s_N\leq U+\epsilon$ for some $N$. Since $s_n$ is decreasing, we actually get $s_n\leq U+\epsilon$ for all $n>N$. So $\lim s_n\leq U$.
Consequently, $\lim s_n=U$, and we see that the definitions are equivalent.
Finally, note that since $s_n$ is decreasing, $\lim_{n\to\infty} s_n=\inf_{n\geq 0} s_n=\inf_{n\geq 0}\sup_{k\geq n}a_k$. This last expression is an alternative definition.
The only significance of the boundedness is to ensure that monotone sequences converge, working in the extended reals has the same effect with less need to separate into cases.
Best Answer
Here's detailed commentary on the proof of Lemma 3.
Lemma 3 (first half). Let $(x_n)$ be a bounded sequence with upper limit $\bar x$. For every $\epsilon>0$ and every $k\in{\mathbb N}$, there exists $n>K$ such that $$|x_n-\bar x|<\epsilon.$$
Proof: Since the sequence $\bar x_k$ decreases to $\bar x$, there exists $K_1$ such that for all $k\ge K_1$, $$ \bar x \le \bar x_k<\bar x + \epsilon.\tag{1} $$ Fix any $k\ge K_1$. Recall that $\bar x_k:=\sup\{x_k, x_{k+1}, x_{k+2},\ldots \}$ by definition. So the left-hand inequality in (1) is a statement that the sup of a certain set exceeds a certain value. Earlier you should have proved a result similar to:
Applying this result to the LHS of (1), we assert that there exists $n\ge k$ such that $$ x_n >\bar x - \epsilon.\tag2 $$ For this $n$ we also have $$ x_n\stackrel{(*)}\le \sup\{x_k, x_{k+1}, x_{k+2},\ldots \}=:\bar x_k\stackrel{(1)}<\bar x + \epsilon,\tag3 $$ where (*) follows from the fact that
To recap, we've established that for every $k\ge K_1$ there exists $n\ge k$ such that $$\bar x -\epsilon\stackrel{(2)}< x_n\stackrel{(3)}<\bar x + \epsilon.\tag4$$ Going back to the statement of the Lemma, the claim is that for any $K$ we can find $n$ such that $n$ exceeds $K$ and $n$ satisfies (4). We haven't even talked about $K$ yet, so this constraint on $n$ hasn't been enforced yet, but we can do so by requiring the $k$ we originally fixed to exceed not just $K_1$ but also $K$.