I paste here a theorem from Mathematical Analysis by S. C. Malik,Savita Aror (see also Google Books here)
Theorem. If $\phi$ is bounded and monotonic in $[a,+\infty)$ and tends to zero at $+\infty$, and $\int_a^X f$ is bounded for $X \geq a$, then $\int_a^\infty f \phi$ is convergent.
As you can see, no continuity is really necessary. Honestly, the continuity assumption appears often for the sake of simplicity. Improper integrals can be defined as limits of Riemann integrals: all you need is local integrability. However, we know that continuity is "almost necessary" to integrate in the sense of Riemann, so teachers do not worry too much about the minimal assumptions under which the theory can be taught.
We can prove that sequences of right- or left-hand Riemann sums will converge for a monotone function with a convergent improper integral.
Suppose WLOG $f:(0,1] \to \mathbb{R}$ is nonnegative and decreasing. Suppose further that there is a singularity at $x =0$ but $f$ is Riemann integrable on $[c,1]$ for $c > 0$ and the improper integral is convergent:
$$\lim_{c \to 0+}\int_c^1 f(x) \, dx = \int_0^1 f(x) \, dx.$$
Take a uniform partition $P_n = (0,1/n, 2/n, \ldots, (n-1)/n,1).$ Since $f$ is decreasing we have
$$\frac1{n}f\left(\frac{k}{n}\right) \geqslant \int_{k/n}^{(k+1)/n}f(x) \, dx \geqslant \frac1{n}f\left(\frac{k+1}{n}\right), $$
and summing over $k = 1,2, \ldots, n-1$
$$\frac1{n}\sum_{k=1}^{n-1}f\left(\frac{k}{n}\right) \geqslant \int_{1/n}^{1}f(x) \, dx \geqslant \frac1{n}\sum_{k=2}^nf\left(\frac{k}{n}\right). $$
Hence,
$$ \int_{1/n}^{1}f(x) \, dx +\frac{1}{n}f(1) \leqslant \frac1{n}\sum_{k=1}^{n}f\left(\frac{k}{n}\right) \leqslant \int_{1/n}^{1}f(x) \, dx+ \frac{1}{n}f \left(\frac{1}{n} \right).$$
Note that as $n \to \infty$ we have $f(1) /n \to 0$ and since the improper integral is convergent,
$$\lim_{n \to \infty} \int_{1/n}^{1}f(x) \, dx = \int_0^1 f(x) \, dx, \\ \lim_{n \to \infty}\frac{1}{n}f \left(\frac{1}{n} \right) = 0.$$
The second limit follows from monotonicity and the Cauchy criterion which implies that for any $\epsilon > 0$ and all $n$ sufficiently large
$$0 \leqslant \frac{1}{n}f \left(\frac{1}{n} \right) \leqslant 2\int_{1/2n}^{1/n}f(x) \, dx < \epsilon.$$
By the squeeze theorem we have
$$\lim_{n \to \infty}\frac1{n}\sum_{k=1}^nf\left(\frac{k}{n}\right) = \int_0^1 f(x) \, dx.$$
This proof can be generalized for non-uniform partitions. For oscillatory functions like $g(x) = \sin(1/x)/x$, the failure of the sequence of right-hand Riemann sums to converge is, non-monotonicity notwithstanding, related to non-convergence as $n \to \infty$ of
$$ \frac{1}{n}g \left(\frac{1}{n} \right) = \sin n. $$
This particular case appears to have been covered nicely by @Daniel Fischer in
Improper integrals and right-hand Riemann sums
Best Answer
Suppose $f : D \subset \mathbb{R}^d \to \mathbb{R}$ is Riemann integrable on every compact rectifiable subset of $D$. The multiple improper integral is defined generally as
$$\int_D f = \lim_{n \to \infty} \int_{D_n}f,$$
where $(D_n)$ is a sequence of compact rectifiable sets such that $D_n \subset \text{int } D_{n+1}$ and $\cup_{n=1}^\infty D_n = D$. The improper integral is well-defined if the limit does not depend on the choice for $(D_n)$.
Under such a definition, it must hold that the improper integral of $f$ over $D$ exists if and only if the improper integral of $|f|$ exists over $D$.
In one-dimension ($d = 1$), the improper integral can be conditionally convergent when defined specifically as a limit of integrals over nested intervals , such as $D_n = [0,n]$ where
$$\int_0^\infty \frac{\sin x}{x} \, dx := \lim_{n \to \infty}\int_0^n \frac{\sin x}{x} \, dx = \frac{\pi}{2}$$
However, even in one-dimension, the more general definition of the improper integral precludes conditional convergence.
For example, consider the following sequence $D_n \subset [0,\infty)$ where each set is a finite union of intervals with gaps,
$$D_n = [0, (2n-1)\pi] \cup \bigcup_{k=n}^{2n}[2k\pi,(2k+1)\pi ]$$
It is easy to show that $D_n \subset D_{n+1}$ for all $n$. Also, for any $c > 0$, there exists $n$ such that $(2n-1)\pi > c$ and $[0,c] \subset D_n$, and this implies $\cup_n D_n = [0,\infty)$.
The integral over $D_n$ is
$$\int_{D_n}\frac{\sin x }{x} \, dx = \int_0^{(2n-1)\pi }\frac{\sin x }{x} \, dx + \sum_{k=n}^{2n} \int_{2k \pi}^{(2k+1) \pi } \frac{\sin x} {x} \, dx,$$
which can be shown to converge to a value greater than $\pi/2 + \log 2 /\pi$.
The first integral on the right-hand side converges to $\pi/2$ and, since $\sin x \geqslant 0$ for $x \in [2k \pi,(2k+1) \pi ]$, it follows that
$$\int_{2k \pi}^{(2k+1) \pi } \frac{\sin x} {x} \, dx > \frac{1}{(2k+1)\pi }\int_{2k \pi}^{(2k+1) \pi } \sin x \, dx = \frac{2}{(2k+1)\pi } > \frac{1}{\pi}\frac{1}{k+1}$$
Thus,
$$\limsup_{n \to \infty}\int_{D_n} \frac{\sin x }{x} \, dx > \frac{\pi}{2} + \lim_{n \to \infty} \frac{1}{\pi} \sum_{k = n}^{2n}\frac{1}{k+1} = \frac{\pi}{2}+ \frac{\log 2}{\pi}$$
There can be no unique value of the limit of the integral over $D_n$ for every choice of sequence $(D_n)$.