Apparently the author only wants to use non-empty open intervals (normally, $\emptyset$ is considered an open interval: this follows from the fact that both open sets and intervals are closed under finite intersection).
So because there are infinitely many terms, he must make the size of these intervals small enough so that their lengths sum to $\varepsilon/2$. How they are placed and whether they overlap doesn't matter at all.
To obtain the bound, he is just applying the definition of $\lambda$ with $I_1=(a-\varepsilon/4,b+\varepsilon/4)$ and $I_n=(-\varepsilon/2^{n+1},\varepsilon/2^{n+1})$ for $n\ge 2$. Because this is a covering of $[a,b]$, it appears in the $\inf$ defining $\lambda$: we have
$$\lambda([a,b])\le \sum_{j\ge 1} |I_j|=|I_1|+\sum_{n\ge 2} \varepsilon\cdot 2^{-n}=(b-a+\varepsilon/2)+\varepsilon/2$$
This problem could have been avoided altogether by allowing empty intervals in the definition of $\lambda$ with $|\emptyset|=0$: this defines the same $\lambda$, because we can always replace $\emptyset$ with arbitrarily small intervals so that the $\inf$ is unchanged.
In the context of $\mathbb R$, explicitly naming the sequence of open intervals is not really necessary. Since every open subset of $\mathbb R$ is the union of such a countable sequence, you can restate the definition of outer measure as follows, first given an arbitrary open set $U = \cup \{ I_k: k \in \mathbb{N} \} \subset \mathbb R$ define,
$$ m^\ast(U) = \sum_{k=0}^\infty \ell(I_k)$$
then, extend the definition of $m^\ast$ to arbitrary subsets $E \subset \mathbb R$ by asserting $m^\ast(E) = $ $\inf \{ m^\ast(U): E \subset U \text{ and } U \text{ is open } \}$.
What does this simplification/restatement accomplish? It enables a cleaner analysis of the process by which the open sets $\mathcal{U} = \{ U \subset \mathbb{R}: U \text{ is open and } E\subset U\}$ are used to find the outer measure of $E$.
Recalling that the infimum $M = \inf \{ m^\ast(U): U \in \mathcal{U}\} = m^\ast(E)$ must satisfy,
For every $\delta > 0$, there exist some $U \in \mathcal{U}$, such that, $ M \le m^\ast(U) < M - \delta$
we can choose/find/fix a sequence of open sets $U_0, \ldots, U_n, \ldots \in \mathcal{U}$ such that, for each $n\ge 0$,
$$ M \le m^\ast(U_n) < M - \frac{1}{2^n}. $$
Letting $V_n = U_0 \cap \ldots \cap U_n$, we get, $V_n \in \mathcal{U}$
(since $E \subset U_0 \cap \ldots \cap U_n = V_n$ and $V_n$ is open.) Moreover, applying the definition of $m^\ast$ for open sets, we have $V_{n+1} \subset V_n \implies$ $m^\ast(V_{n+1}) \le m^\ast(V_n)$,and
$$ M \le m^\ast(V_n) \le \min \{ m^\ast(U_0), \ldots, m^\ast(U_n) \} < M - \frac{1}{2^n}.$$
Putting this all together, we've extracted a descending sequence $V_n$ of open sets containing $E$, whose measures converge to the value of the outer measure $M = m^\ast(E)$ of $E$.
You can think of the sets $V_k$ as approximating $E$ by shrinking and progressively discarding non-essential segments. Moreover, capturing that processing is the whole idea behind how the outer measure is defined.
Best Answer
Since closed intervals $A$ of finite length are compact, any open cover of $A$ admits a finite subcover. So we can restrict our attention to finite covers. It is not necessary to do this, it's just more convenient. In the comments, we developed a guess saying that one could use connectedness instead of compactness to prove this.
Having established that $m^*((a,b)) = \ell((a,b))$, one way to prove that $m^*(A) = \ell(A)$ without using compactness would be as follows. Since the set consisting of the endpoints of $A$ has measure zero (it is countable), removing them from $A$ yields a set of the same measure. Since the set obtained is the interior of $A$ we can conclude by just showing that open intervals have measure equal to their length. (Note that Royden does not prove this first, but I'm sure it can be done somehow.)