Probably the best way to talk about this is with an example. Let $f_n:[0, 1] \to \mathbb{R}$ where $$ f(x) = \begin{cases} 1 &\mbox{if } x 2^n \in \mathbb{N} \\
0 & \mbox{otherwise. }\end{cases} \pmod{2} $$
Now, you can see that each $f_n$ is integrable, in both the Riemannian and Lebesgue sense. The limit of the sequence is the function that is $1$ for every value with a terminating binary representation and $0$ elsewhere. This is not Riemann integrable as the lower sums and upper sums don't converge to the same value: the lower sums are always $0$ and the upper sums are always $1$, no matter what partition we choose. However, the limit is Lebesgue integrable, with an integral of $0$.
Now, if I were to ask you "what is the area under the curve?" I'd hope that we could agree that its $0$. To paint it I'd have to make a whole lot of lines, but the lines have $0$ width, so... I feel like the area is $0$. Riemann integration just isn't deep enough to see this fact. So, I'd say they're both area, but one is a better, broader idea of area.
Regarding your final question, can the upper bound differ from the lower? Try to relate the two, you should be pleasantly surprised.
Your proof is valid for a bounded function defined on the closed, bounded interval $[a,b]$, despite the apparent simplicity. It also relies on the fact that the Riemann and Lebesgue integrals are the same for step functions, as mentioned by Tony Piccolo.
The other proof you mention is also valid but takes you on a more roundabout path because it strings together a number of results, each of which is not altogether trivial to prove.
A bounded, measurable function defined on a set of finite
measure is Lebesgue integrable.
If a sequence of measurable functions converges almost everywhere
to $f$, then the limit function $f$ is measurable.
If $f$ is Riemann integrable on $[a,b]$, then there exists a
sequence of simple (measurable) functions converging almost everywhere
to $f$.
Adding even more complexity, the proof of the third statement that I know also uses the fact that the set of discontinuities of a Riemann integrable function must be of measure zero. It begins by constructing a partition of $[a,b]$ with dyadic intervals:
$$I_{n,k} = \begin{cases}[a + (b-a)\frac{k-1}{2^n}, a + (b-a)\frac{k}{2^n})\,\,\, k = 1 , \ldots, 2^n -1 \\ [a + (b-a)\frac{2^n-1}{2^n}, b] \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, k = 2^n\end{cases} $$
With $m_k = \inf_{I_{n,k}}f(x)$, we can construct the sequence of simple functions
$$\phi_n(x) = \sum_{k=1}^{2^n} m_k \, \chi_{I_{n,k}}(x)$$
The sequence is increasing and, since $f$ is bounded, by monotonicity is convergent to some function $\phi$ such that $\phi_n(x) \uparrow \phi(x) \leqslant f(x).$ If $x \in I_{n,k}$ then the oscillation of $f$ over that interval satisfies
$$\sup_{u,v \in I_{n,k}}|f(u) - f(v)| \geqslant f(x) - \phi_n(x) \geqslant f(x) - \phi(x).$$
Thus, $f(x) \neq \phi(x)$ only at points where $f$ is not continuous, which must belong to a measure zero set if $f$ is Riemann integrable. Therefore, $f$ is almost everywhere the limit of a sequence of simple functions and is measurable.
I would agree that the second approach is "pretty complicated". However, I think it is not uncommon to find theorems with a variety of proofs ranging in complexity.
Best Answer
If $f$ is bounded and measurable on a set $E \subset \mathbb{R}$ then for any $\epsilon>0$ there are simple functions $s_\epsilon$ and $t_\epsilon$ such that $s_\epsilon \leqslant f \leqslant t_\epsilon$ and $t_\epsilon - s_\epsilon < \epsilon$. This is not difficult to prove and $E$ need not be bounded.
If, in fact, $E$ is bounded, then by the basic properties of integrals of simple functions (monotone and linear) we have
$$0 \leqslant \inf\left\{ \int_E t \, d\mu : t \geqslant f, t \text{ simple } \right\} - \sup\left\{ \int_E s \, d\mu : 0 \leqslant s \leqslant f, s \text{ simple } \right\} \\ \leqslant \int_E t_\epsilon \, d\mu - \int_E s_\epsilon \, d\mu = \int_E (t_\epsilon - s_\epsilon) \, d\mu < \epsilon \, \mu(E).$$
Since this is true for any $\epsilon > 0$ we always have
$$\inf\left\{ \int_E t \, d\mu : t \geqslant f, t \text{ simple } \right\} = \sup\left\{ \int_E s \, d\mu : 0 \leqslant s \leqslant f, s \text{ simple } \right\}, $$
and either the $\inf$ or $\sup$ defines the Lebesgue integral. The Riemann integral need not exist.
If, however, $f$ is nonnegative and either $f$ is unbounded or $\mu(E) = \infty$ or both, then the integral is defined as
$$\int_E f = \sup \left\{\int_E g: 0 \leqslant g \leqslant f, \, g \text{ bounded, measurable, and of finite support} \right\}.$$
Such $\int_E g$ are defined in the first part and this can be extended in general using $f = f^+ - f^-.$