Even with the Riemann Integral, we do not usually use the definition (as a limit of Riemann sums, or by verifying that the limit of the upper sums and the lower sums both exist and are equal) to compute integrals. Instead, we use the Fundamental Theorem of Calculus, or theorems about convergence. The following are taken from Frank E. Burk's A Garden of Integrals, which I recommend. One can use these theorems to compute integrals without having to go down all the way to the definition (when they are applicable).
Theorem (Theorem 3.8.1 in AGoI; Convergence for Riemann Integrable Functions) If $\{f_k\}$ is a sequence of Riemann integrable functions converging uniformly to the function $f$ on $[a,b]$, then $f$ is Riemann integrable on $[a,b]$ and
$$R\int_a^b f(x)\,dx = \lim_{k\to\infty}R\int_a^b f_k(x)\,dx$$
(where "$R\int_a^b f(x)\,dx$" means "the Riemann integral of $f(x)$").
Theorem (Theorem 3.7.1 in AGoI; Fundamental Theorem of Calculus for the Riemann Integral) If $F$ is a differentiable function on $[a,b]$, and $F'$ is bounded and continuous almost everywhere on $[a,b]$, then:
- $F'$ is Riemann-integrable on $[a,b]$, and
- $\displaystyle R\int_a^x F'(t)\,dt = F(x) - F(a)$ for each $x\in [a,b]$.
Likewise, for Riemann-Stieltjes, we don't usually go by the definition; instead we try, as far as possible, to use theorems that tell us how to evaluate them. For example:
Theorem (Theorem 4.3.1 in AGoI) Suppose $f$ is continuous and $\phi$ is differentiable, with $\phi'$ being Riemann integrable on $[a,b]$. Then the Riemann-Stieltjes integral of $f$ with respect to $\phi$ exists, and
$$\text{R-S}\int_a^b f(x)d\phi(x) = R\int_a^b f(x)\phi'(x)\,dx$$
where $\text{R-S}\int_a^bf(x)d\phi(x)$ is the Riemann-Stieltjes integral of $f$ with respect to $d\phi(x)$.
Theorem (Theorem 4.3.2 in AGoI) Suppose $f$ and $\phi$ are bounded functions with no common discontinuities on the interval $[a,b]$, and that the Riemann-Stieltjes integral of $f$ with respect to $\phi$ exists. Then the Riemann-Stieltjes integral of $\phi$ with respect to $f$ exists, and
$$\text{R-S}\int_a^b \phi(x)df(x) = f(b)\phi(b) - f(a)\phi(a) - \text{R-S}\int_a^bf(x)d\phi(x).$$
Theorem. (Theorem 4.4.1 in AGoI; FTC for Riemann-Stieltjes Integrals) If $f$ is continuous on $[a,b]$ and $\phi$ is monotone increasing on $[a,b]$, then $$\displaystyle \text{R-S}\int_a^b f(x)d\phi(x)$$
exists. Defining a function $F$ on $[a,b]$ by
$$F(x) =\text{R-S}\int_a^x f(t)d\phi(t),$$
then
- $F$ is continuous at any point where $\phi$ is continuous; and
- $F$ is differentiable at each point where $\phi$ is differentiable (almost everywhere), and at such points $F'=f\phi'$.
Theorem. (Theorem 4.6.1 in AGoI; Convergence Theorem for the Riemann-Stieltjes integral.) Suppose $\{f_k\}$ is a sequence of continuous functions converging uniformly to $f$ on $[a,b]$ and that $\phi$ is monotone increasing on $[a,b]$. Then
The Riemann-Stieltjes integral of $f_k$ with respect to $\phi$ exists for all $k$; and
The Riemann-Stieltjes integral of $f$ with respect to $\phi$ exists; and
$\displaystyle \text{R-S}\int_a^b f(x)d\phi(x) = \lim_{k\to\infty} \text{R-S}\int_a^b f_k(x)d\phi(x)$.
One reason why one often restricts the Riemann-Stieltjes integral to $\phi$ of bounded variation is that every function of bounded variation is the difference of two monotone increasing functions, so we can apply theorems like the above when $\phi$ is of bounded variation.
For the Lebesgue integral, there are a lot of "convergence" theorems: theorems that relate the integral of a limit of functions with the limit of the integrals; these are very useful to compute integrals. Among them:
Theorem (Theorem 6.3.2 in AGoI) If $\{f_k\}$ is a monotone increasing sequence of nonnegative measurable functions converging pointwise to the function $f$ on $[a,b]$, then the Lebesgue integral of $f$ exists and
$$L\int_a^b fd\mu = \lim_{k\to\infty} L\int_a^b f_kd\mu.$$
Theorem (Lebesgue's Dominated Convergence Theorem; Theorem 6.3.3 in AGoI) Suppose $\{f_k\}$ is a sequence of Lebesgue integrable functions ($f_k$ measurable and $L\int_a^b|f_k|d\mu\lt\infty$ for all $k$) converging pointwise almost everywhere to $f$ on $[a,b]$. Let $g$ be a Lebesgue integrable function such that $|f_k|\leq g$ on $[a,b]$ for all $k$. Then $f$ is Lebesgue integrable on $[a,b]$ and
$$L\int_a^b fd\mu = \lim_{k\to\infty} L\int_a^b f_kd\mu.$$
Theorem (Theorem 6.4.2 in AGoI) If $F$ is a differentiable function, and the derivative $F'$ is bounded on the interval $[a,b]$, then $F'$ is Lebesgue integrable on $[a,b]$ and
$$L\int_a^x F'd\mu = F(x) - F(a)$$
for all $x$ in $[a,b]$.
Theorem (Theorem 6.4.3 in AGoI) If $F$ is absolutely continuous on $[a,b]$, then $F'$ is Lebesgue integrable and
$$L\int_a^x F'd\mu = F(x) - F(a),\qquad\text{for }x\text{ in }[a,b].$$
Theorem (Theorem 6.4.4 in AGoI) If $f$ is continuous and $\phi$ is absolutely continuous on an interval $[a,b]$, then the Riemann-Stieltjes integral of $f$ with respect to $\phi$ is the Lebesgue integral of $f\phi'$ on $[a,b]$:
$$\text{R-S}\int_a^b f(x)d\phi(x) = L\int_a^b f\phi'd\mu.$$
For Lebesgue-Stieltjes Integrals, you also have an FTC:
Theorem. (Theorem 7.7.1 in AGoI; FTC for Lebesgue-Stieltjes Integrals) If $g$ is a Lebesgue measurable function on $R$, $f$ is a nonnegative Lebesgue integrable function on $\mathbb{R}$, and $F(x) = L\int_{-\infty}^xd\mu$, then
- $F$ is bounded, monotone increasing, absolutely continuous, and differentiable almost everywhere with $F' = f$ almost everywhere;
- There is a Lebesgue-Stieltjes measure $\mu_f$ so that, for any Lebesgue measurable set $E$, $\mu_f(E) = L\int_E fd\mu$, and $\mu_f$ is absolutely continuous with respect to Lebesgue measure.
- $\displaystyle \text{L-S}\int_{\mathbb{R}} gd\mu_f = L\int_{\mathbb{R}}gfd\mu = L\int_{\mathbb{R}} gF'd\mu$.
The Henstock-Kurzweil integral likewise has monotone convergence theorems (if $\{f_k\}$ is a monotone sequence of H-K integrable functions that converge pointwise to $f$, then $f$ is H-K integrable if and only if the integrals of the $f_k$ are bounded, and in that case the integral of the limit equals the limit of the integrals); a dominated convergence theorem (very similar to Lebesgue's dominated convergence); an FTC that says that if $F$ is differentiable on $[a,b]$, then $F'$ is H-K integrable and
$$\text{H-K}\int_a^x F'(t)dt = F(x) - F(a);$$
(this holds if $F$ is continuous on $[a,b]$ and has at most countably many exceptional points on $[a,b]$ as well); and a "2nd FTC" theorem.
Although this doesn't fully satisfy all your requests, I think it will be useful to state a theorem which appears in some similar form in Folland's Real Analysis (I believe 2.28, though I might be wrong):
Theorem: Suppose that $f:[a,b] \to \Bbb{R}$ is bounded. Then:
1) If $f$ if Riemann integrable, then $f$ is Lebesgue measurable and the Riemann integral $\int_a^b f(x) \, dx$ equals the Lebesgue integral $\int_{[a,b]} f \, d\mu$ (where $\mu$ is Lebesgue measure).
2) Further $f$ is Riemann integrable if and only if the set of discontinuities of $f$ is a $\mu$-null set.
End Theorem
Let me make another point about a difference in Riemann and Lebesgue integration from a geometric standpoint. If you notice, the Lebesgue integral $\int_{[a,b]} f \, d\mu$ has no "orientation" with respect to the interval $[a,b]$, whereas the Riemann integral $\int_a^b f(x)\,dx$ we interpret as "the integral of $f$ from $a$ to $b$" and, in fact, we have $\int_a^b = - \int_b^a$ for Riemann integrals. To further investigate this point, I'd suggest looking into differential geometry; in this context $dx$ would play the role of the "Riemann volume form" on $\Bbb{R}$ which depends on this orientation, whereas $d\mu$ is an honest measure and doesn't require orientation.
Best Answer
As copper.hat pointed out $C_c(\mathbb{R})$ is dense in $L^1(\mathbb{R})$, so we had to define such an integral for continuous functions with compact support.
The idea to define integral as linear functional is due to Daniell.
As for a good introduction to integration theory from functional analysis point of view see chapter III of An Introduction To Abstract Harmonic Analysis, Loomis,Lynn H..