Of course, the correct definition of coherence is that in your Question 2. It just so happens that for a sheaf of modules on a scheme it is equivalent to the easier one.
As far as a I know, the notion of coherence is mostly used when one has a sheaf of ring (often non-commutative), different from the structure sheaf. So, for example, the sheaf of D-modules on a smooth variety is not noetherian, but it is a coherent sheaf of rings; this is a very important fact.
There are schemes whose structure sheaf is not coherent, and those are a bit of a mess; for example, the locally finitely presented quasi-coherent sheaves do not form an abelian category. However, in most cases one is not usually bothered by them.
For example, in setting up a moduli problem, it is quite useful to consider non-noetherian base schemes, because the category of locally noetherian schemes has problems (for example, is not closed under fibered products). For example, if $X$ is a projective scheme over a field $k$, and $F$ is a coherent sheaf on $X$, one defined the Quot functor from schemes over $k$ to sets by sending each (possibly non-noetherian scheme) $k$-scheme $T$ into the set of finitely presented quotients of the pullback $F_T$ of $F$ to $T$ that are flat over $T$. Of course, when you actually prove something, one uses that fact that locally on $T$ any finitely presented sheaf comes from one defined a finitely generated $k$-algebra, and works with that, free to use all the results that hold in the noetherian context. Thus, in practice most of the time you don't need to do anything with non-noetherian schemes.
Best Answer
Perhaps what you need is an action of of a tt-category (in your case $D(R)$) on the category $D_{qc}(X)$ which gives a support in $S$ to quasi-coherent complexes over $X$. This is discussed with greater generality in the paper:
Stevenson, Greg: Support theory via actions of tensor triangulated categories. J. Reine Angew. Math. 681 (2013), 219–254.
(also available in arXiv).