Assume that $g$ is increasing. I suppose that you know that $f\in\mathcal{R}([a,b],g)$ iff $f$ satisfies the Riemann's condition. The Riemann's condition says:
$f$ satisfies the Riemann's condition respect to $g$ in $[a,b]$ if for every $\epsilon\gt 0$, there exist a partition $P_\epsilon$ of $[a,b]$ such that if $P$ is a refinement of $P_\epsilon$ then $$0\leq U(P,f,g)-L(P,f,g)\lt \epsilon,$$
where $U(P,f,g)$ and $L(P,f,g)$ are the upper and lower Riemann-Stieltjes sums respectively.
With this and the hint in the comments the result holds.
Perhaps the chapter 7 of Mathematical Analysis of Tom M. Apostol can be useful to you.
Imo, the point of the Riemann-Stieltjes integral is not that "We can now integrate more functions, which makes us happy." (btw. I'm doubtful this is actually true). The point is, that we now have a more general way of expressing certain things as an integrals, or a clearer view of what an integral naturally should be.
For example: We may think of the expression $\int_a^b \rho \, dx$ as the total mass of some segment $[a,b]$, where $\rho: [a,b] \to \mathbb R$ is the mass density on $[a,b]$. In this situation, the Riemann integral is sufficient.
But what the Riemann integral allows us to do may not be sufficient in some cases. Sometimes, we really would like to talk about point-like objects, such as point-masses. Intuitively, the density $\rho$ of a point-mass situated at $x=0$, say, should be zero everywhere, except at $0$. At the same time, we would like to have
$$\int_{\mathbb R} \rho \, dx = \mathrm{mass}.$$
Now, with a Riemann integral, we can never achieve this. There simply is no function $\rho$, such that $\rho(x)=0$ for $x\ne 0$, and yet $\int_{\mathbb R}\rho \, dx \ne 0$.
Bring in the Riemann-Stieltjes integral: Let $F$ be given by $F(x) = 0$, if $x<0$, and $F(x) = \mathrm{mass}$, if $x\ge 0$. Then $F(x)$ is the total mass on $(-\infty, x]$. Now, $F$ does not have a density of the form $\rho\, dx$, but it does have a density $dF$ in the R-S sense. And this does what we want: We have
$$\int_{-\infty}^x \, dF = \begin{cases} 0 & x< 0\\ \mathrm{mass} & x\ge 0.\end{cases}$$
So, using R-S integrals, we have gained a generalized notion of "density". In fact, we learn from this that when speaking about the density of something, we should not think of it in terms of a function $\rho$, but rather in terms of an expression like $\rho dx$, or $dF$. Said differently, we see that the Riemann integral, or the $dx$ density, is just a very special way of measuring something, which pertains to certian situations but not to others. There are situations, where measurement should be done in a way which is not captured by the Riemann integral.
This realization may have been the first step in the direction of the notion of a measure. A concept which underlies much of modern analysis.
It is in this way, that the Riemann-Stieltjes integral generalizes the Riemann integral.
Best Answer
A counterexample is any Riemann integrable function $f:[a,b] \mapsto \mathbb{R}$ which is discontinuous at some point in the interval and $g = f$.
It is proved here that the Riemann-Stieltjes integral of $f$ with respect to $g$ does not exist if the functions are both discontinuous from the left at a point or both discontinuous from the right. (It is possible for the integral to exist, however, if they are both discontinuous only from different directions.)
Hence, if $f$ is Riemann integrable but discontinuous in any way at a point in $[a,b]$, then -- when $f$ is both integrand and integrator -- a common discontinuity from one side is always shared and $\int_a^b f \, df$ fails to exist.