You should get that the answer is $1/2$, immediately from integration by parts. See this, in particular the sections "Integration by parts" and "Related concepts". Further, see this for a proof of the integration by parts formula.
EDIT: Explicitly, since $f$ is continuous and non-decreasing, and is constant on $\mathbb{R}-[0,1]$, $\int_{\mathbb R} {f(x)du(x)} = \int_0^1 {f(x)df(x)} $, and it holds
$$
\int_0^1 {f(x)df(x)} = f(1)f(1) - f(0)f(0) - \int_0^1 {f(x)df(x)}
$$
(integration by parts). Since $f(0)=0$ and $f(1)=1$, it thus follows that $\int_{\mathbb R} {f(x)du(x)} = 1/2$.
EDIT: More generally, if $F$ is any continuous distribution function (the Cantor function $f$ is a particular example), then $\int_{\mathbb R} {F(x)dF(x)} = 1/2$. As before, this can be proved using integration by parts, which is allowed since $F$ is continuous and non-decreasing. (Under certain conditions, this also follows from a change of variable.) Indeed, for any $a < b$,
$$
\int_a^b {F(x)dF(x)} = F(b)F(b) - F(a)F(a) - \int_a^b {F(x)dF(x)}.
$$
Hence the result follows by letting $a \to -\infty$ and $b \to \infty$.
Another way to obtain the general result is as follows. Let $X$ be an arbitrary random variable with continuous distribution function $F$. Then, $\int_{\mathbb R} {F(x)dF(x)}$ expresses the expectation of the random variable $F(X)$. As is well known, in this case $F(X) \sim {\rm uniform}(0,1)$. Hence, $\int_{\mathbb R} {F(x)dF(x)} = 1/2$.
Remark. With $f$ as above, integration by parts gives
$$
\int_0^1 {xdf(x)} = xf(x)\big|_0^1 - \int_0^1 {f(x)dx} = 1 - 1/2 = 1/2.
$$
The left-hand side, $\int_0^1 {xdf(x)}$, expresses the expectation of the Cantor distribution.
Even with the Riemann Integral, we do not usually use the definition (as a limit of Riemann sums, or by verifying that the limit of the upper sums and the lower sums both exist and are equal) to compute integrals. Instead, we use the Fundamental Theorem of Calculus, or theorems about convergence. The following are taken from Frank E. Burk's A Garden of Integrals, which I recommend. One can use these theorems to compute integrals without having to go down all the way to the definition (when they are applicable).
Theorem (Theorem 3.8.1 in AGoI; Convergence for Riemann Integrable Functions) If $\{f_k\}$ is a sequence of Riemann integrable functions converging uniformly to the function $f$ on $[a,b]$, then $f$ is Riemann integrable on $[a,b]$ and
$$R\int_a^b f(x)\,dx = \lim_{k\to\infty}R\int_a^b f_k(x)\,dx$$
(where "$R\int_a^b f(x)\,dx$" means "the Riemann integral of $f(x)$").
Theorem (Theorem 3.7.1 in AGoI; Fundamental Theorem of Calculus for the Riemann Integral) If $F$ is a differentiable function on $[a,b]$, and $F'$ is bounded and continuous almost everywhere on $[a,b]$, then:
- $F'$ is Riemann-integrable on $[a,b]$, and
- $\displaystyle R\int_a^x F'(t)\,dt = F(x) - F(a)$ for each $x\in [a,b]$.
Likewise, for Riemann-Stieltjes, we don't usually go by the definition; instead we try, as far as possible, to use theorems that tell us how to evaluate them. For example:
Theorem (Theorem 4.3.1 in AGoI) Suppose $f$ is continuous and $\phi$ is differentiable, with $\phi'$ being Riemann integrable on $[a,b]$. Then the Riemann-Stieltjes integral of $f$ with respect to $\phi$ exists, and
$$\text{R-S}\int_a^b f(x)d\phi(x) = R\int_a^b f(x)\phi'(x)\,dx$$
where $\text{R-S}\int_a^bf(x)d\phi(x)$ is the Riemann-Stieltjes integral of $f$ with respect to $d\phi(x)$.
Theorem (Theorem 4.3.2 in AGoI) Suppose $f$ and $\phi$ are bounded functions with no common discontinuities on the interval $[a,b]$, and that the Riemann-Stieltjes integral of $f$ with respect to $\phi$ exists. Then the Riemann-Stieltjes integral of $\phi$ with respect to $f$ exists, and
$$\text{R-S}\int_a^b \phi(x)df(x) = f(b)\phi(b) - f(a)\phi(a) - \text{R-S}\int_a^bf(x)d\phi(x).$$
Theorem. (Theorem 4.4.1 in AGoI; FTC for Riemann-Stieltjes Integrals) If $f$ is continuous on $[a,b]$ and $\phi$ is monotone increasing on $[a,b]$, then $$\displaystyle \text{R-S}\int_a^b f(x)d\phi(x)$$
exists. Defining a function $F$ on $[a,b]$ by
$$F(x) =\text{R-S}\int_a^x f(t)d\phi(t),$$
then
- $F$ is continuous at any point where $\phi$ is continuous; and
- $F$ is differentiable at each point where $\phi$ is differentiable (almost everywhere), and at such points $F'=f\phi'$.
Theorem. (Theorem 4.6.1 in AGoI; Convergence Theorem for the Riemann-Stieltjes integral.) Suppose $\{f_k\}$ is a sequence of continuous functions converging uniformly to $f$ on $[a,b]$ and that $\phi$ is monotone increasing on $[a,b]$. Then
The Riemann-Stieltjes integral of $f_k$ with respect to $\phi$ exists for all $k$; and
The Riemann-Stieltjes integral of $f$ with respect to $\phi$ exists; and
$\displaystyle \text{R-S}\int_a^b f(x)d\phi(x) = \lim_{k\to\infty} \text{R-S}\int_a^b f_k(x)d\phi(x)$.
One reason why one often restricts the Riemann-Stieltjes integral to $\phi$ of bounded variation is that every function of bounded variation is the difference of two monotone increasing functions, so we can apply theorems like the above when $\phi$ is of bounded variation.
For the Lebesgue integral, there are a lot of "convergence" theorems: theorems that relate the integral of a limit of functions with the limit of the integrals; these are very useful to compute integrals. Among them:
Theorem (Theorem 6.3.2 in AGoI) If $\{f_k\}$ is a monotone increasing sequence of nonnegative measurable functions converging pointwise to the function $f$ on $[a,b]$, then the Lebesgue integral of $f$ exists and
$$L\int_a^b fd\mu = \lim_{k\to\infty} L\int_a^b f_kd\mu.$$
Theorem (Lebesgue's Dominated Convergence Theorem; Theorem 6.3.3 in AGoI) Suppose $\{f_k\}$ is a sequence of Lebesgue integrable functions ($f_k$ measurable and $L\int_a^b|f_k|d\mu\lt\infty$ for all $k$) converging pointwise almost everywhere to $f$ on $[a,b]$. Let $g$ be a Lebesgue integrable function such that $|f_k|\leq g$ on $[a,b]$ for all $k$. Then $f$ is Lebesgue integrable on $[a,b]$ and
$$L\int_a^b fd\mu = \lim_{k\to\infty} L\int_a^b f_kd\mu.$$
Theorem (Theorem 6.4.2 in AGoI) If $F$ is a differentiable function, and the derivative $F'$ is bounded on the interval $[a,b]$, then $F'$ is Lebesgue integrable on $[a,b]$ and
$$L\int_a^x F'd\mu = F(x) - F(a)$$
for all $x$ in $[a,b]$.
Theorem (Theorem 6.4.3 in AGoI) If $F$ is absolutely continuous on $[a,b]$, then $F'$ is Lebesgue integrable and
$$L\int_a^x F'd\mu = F(x) - F(a),\qquad\text{for }x\text{ in }[a,b].$$
Theorem (Theorem 6.4.4 in AGoI) If $f$ is continuous and $\phi$ is absolutely continuous on an interval $[a,b]$, then the Riemann-Stieltjes integral of $f$ with respect to $\phi$ is the Lebesgue integral of $f\phi'$ on $[a,b]$:
$$\text{R-S}\int_a^b f(x)d\phi(x) = L\int_a^b f\phi'd\mu.$$
For Lebesgue-Stieltjes Integrals, you also have an FTC:
Theorem. (Theorem 7.7.1 in AGoI; FTC for Lebesgue-Stieltjes Integrals) If $g$ is a Lebesgue measurable function on $R$, $f$ is a nonnegative Lebesgue integrable function on $\mathbb{R}$, and $F(x) = L\int_{-\infty}^xd\mu$, then
- $F$ is bounded, monotone increasing, absolutely continuous, and differentiable almost everywhere with $F' = f$ almost everywhere;
- There is a Lebesgue-Stieltjes measure $\mu_f$ so that, for any Lebesgue measurable set $E$, $\mu_f(E) = L\int_E fd\mu$, and $\mu_f$ is absolutely continuous with respect to Lebesgue measure.
- $\displaystyle \text{L-S}\int_{\mathbb{R}} gd\mu_f = L\int_{\mathbb{R}}gfd\mu = L\int_{\mathbb{R}} gF'd\mu$.
The Henstock-Kurzweil integral likewise has monotone convergence theorems (if $\{f_k\}$ is a monotone sequence of H-K integrable functions that converge pointwise to $f$, then $f$ is H-K integrable if and only if the integrals of the $f_k$ are bounded, and in that case the integral of the limit equals the limit of the integrals); a dominated convergence theorem (very similar to Lebesgue's dominated convergence); an FTC that says that if $F$ is differentiable on $[a,b]$, then $F'$ is H-K integrable and
$$\text{H-K}\int_a^x F'(t)dt = F(x) - F(a);$$
(this holds if $F$ is continuous on $[a,b]$ and has at most countably many exceptional points on $[a,b]$ as well); and a "2nd FTC" theorem.
Best Answer
It should just be a consequence of
$\int_a^bf(x)dg(x) = f(b)g(b)-f(a)g(a)-\int_a^b g(x)df(x) $
which can be seen as integration by parts for the Stieltjes integral.
Not sure about the Lebesgue-Stieltjes integral myself, but it's not too hard to show that this holds for the Riemann-Stieltjes integral: you just have to rollback to the definition and rearrange the terms of the sum in order to get something like
$\sum f(c_i)(g(x_{i+1})-g(x_i)) = f(a)g(a)-f(b)g(b)- \sum g(x_i)(f(c_{i+1})-f(c_i))$
From this you should be able to give sense of the answer you got:
$\int_a^b \varphi(x)d\varphi(x) = \varphi(b)\varphi(b)-\varphi(a)\varphi(a)-\int_a^b \varphi(x)d\varphi(x) = 1 - \int_a^b \varphi(x)d\varphi(x)$