Consider a partition $a = x_0 < x_1 < \ldots < x_n = b$ and for a subinterval $I_j = [x_{j-1},x_j]$ define
$$D_{j}(g) = \sup_{x \in I_j}g(x) - \inf_{x \in I_j}g(x) = \sup_{x,y \in I_j}|g(x) - g(y)| \\ D_{j}(f \circ g ) = \sup_{x \in I_j}f(g(x)) - \inf_{x \in I_j}f(g(x)) = \sup_{x,y \in I_j}|f(g(x)) - f(g(y))|$$
Since $f$ is continuous it is uniformly continuous and bounded (by extension if necessary) on a closed interval $[c,d]$ such that $g([a,b]) \subset [c,d].$
Hence, $|f(x)| \leqslant M$ for $x \in [c,d]$ and for every $\epsilon >0$ there exists $\delta > 0$ such that if $|x_1 - x_2| < \delta$ then $|f(x_1) - f(x_2)| < \epsilon/(2(b-a))$ .
Since $g$ is integrable, if the partition norm $\|P\|$ is sufficiently small we have
$$U(P,g) - L(P,g) = \sum_{j=1}^n D_j(g) (x_j - x_{j-1}) < \frac{ \delta \epsilon}{4M}.$$
We can split the upper-lower sum difference $U(P,f \circ g) - L(P, f \circ g)$ into two sums as given by
$$\tag{1}U(P,f \circ g) - L(P, f \circ g) = \sum_{D_j(g) \geqslant \delta} D_j(f \circ g)(x_j - x_{j-1}) + \sum_{D_j(g) < \delta} D_j(f \circ g)(x_j - x_{j-1})$$
In the second sum on the RHS of (1) we have $D_j(f \circ g) < \epsilon/(2(b-a)$ since by uniform continuity $D_j(g) < \delta \implies |g(x) - g(y)| < \delta \implies |f(g(x)) - f(g(y))| < \epsilon/(2(b-a)$ for all $x,y \in I_j$.
Thus,
$$\tag{2}\sum_{D_j(g) < \delta} D_j(f \circ g)(x_j - x_{j-1}) < \frac{\epsilon}{2}$$
Considering the first sum on the RHS of (1), first note that
$$\sum_{D_j(g) \geqslant \delta} (x_j - x_{j-1}) \\ < \delta^{-1}\sum_{D_j(g) \geqslant \delta} D_j(g)(x_j - x_{j-1}) < \delta^{-1} [U(P,g) - L(P,g)] < \delta^{-1} \frac{\delta \epsilon}{4M} = \frac{\epsilon}{4M} .$$
Hence,
$$\tag{3}\sum_{D_j(g) \geqslant \delta} D_j(f \circ g)(x_j - x_{j-1}) < \sum_{D_j(g) \geqslant \delta} 2M(x_j - x_{j-1}) < \frac{\epsilon}{2}.$$
From (1), (2) and (3) we obtain
$$U(P,f \circ g) - L(P, f \circ g) < \epsilon,$$
and conclude that $f \circ g$ is integrable.
Best Answer
A "proper" Riemann integral works under two assumptions:
1) interval of integration is bounded
2) function being integrated is bounded in that interval.
If any of these assumptions are not applicable then we have to introduce "improper" Riemann integrals as limits of suitable "proper" Riemann Integrals. The first theorem deals with "proper" Riemann integrals where the function is bounded and interval of integration is also bounded.
The second result (in question) deals with the case where interval of integration is bounded, but the function is unbounded in that interval. Note that in this case the function tends to $\infty$ at the end point of the interval. Hence a way to fix this problem is to replace interval $[a, b]$ with $[a + d, b]$ (assuming that function becomes unbounded at the end point $a$) and take limit of integral on $[a + d, b]$ as $d \to 0^{+}$. If the function was unbounded at $b$ then we need to consider the interval $[a, b - d]$ and take limit of integral as $d \to 0^{+}$.
Coming back to your example $(\sin x)/x$, we can see that it is bounded on whole of $\mathbb{R}$, so any integral $$\int_{a}^{b}\frac{\sin x}{x}\,dx$$ exists as a "proper" Riemann integral. The second definition of "improper" integrals does not apply to it. On the other hand we have the standard result $$\frac{\pi}{2} = \int_{0}^{\infty}\frac{\sin x}{x}\,dx = \lim_{x \to \infty}\int_{0}^{x}\frac{\sin t}{t}\,dt$$ so that the integral $\int_{0}^{\infty}((\sin x)/x)\,dx$ exists as an "improper" Riemann integral (this is the case where interval of integration is unbounded, but the function is bounded).
Your textbook probably has a typo when it is trying to introduce the improper integrals. It should say that $f$ is continuous on $(a, b]$ and unbounded on $(a, b]$.
There is another theorem which deals with the case when $f$ is bounded on $(a, b]$ which sort of says that if the function is bounded then taking limits (as in case of improper integrals) does not lead to anything new.
Theorem: If $f$ is bounded on $[a, b]$ and $$\int_{a + d}^{b}f(x)\,dx$$ exists for all $d \in (0, b - a)$ then $$\int_{a}^{b}f(x)\,dx$$ exists as a "proper" Riemann integral. Further $$\lim_{d \to 0^{+}}\int_{a + d}^{b}f(x)\,dx = \int_{a}^{b}f(x)\,dx$$
For a proof of the above theorem see this question. From the above theorem we see that there is no need to introduce the concept of "improper" Riemann integrals when the function and the interval are both bounded.
Note: The use of adjective "proper" here is invented for this answer in order to contrast the Riemann integral with "improper" Riemann integrals.