Yep, a quasi-compact morphism of schemes (resp. locally noetherian schemes) is universally closed if and only the existence part of the valuative criterion holds for complete valuation rings (resp. complete DVRs) with algebraically closed residue field.
This is in EGA (see II.7.3.8 and the remark II.7.3.9). Note that the separated hypothesis is not necessary there; for the valuative criterion of properness one needs to require that the morphism is quasi-separated.
This holds more generally for Artin stacks if one allows a field extension of the fraction field of the valuation ring (see LMB 7.3).
If you have a separated, finite type morphism of schemes which are finite type over a field (some more general bases are also valid), then you can get this reduction from Chow's lemma: every finite type, separated scheme over a field is the target of a projective, birational morphism whose domain is quasi-projective.
First of all, properness is local on the target. Thus, you may replace your original target with a covering by open affines. So assume that the domain and target are both separated.
Next apply Chow's lemma to the domain. This gives a new morphism with the same target as the original morphism, but the domain of the new morphism is quasi-projective. It is easy to see that the new morphism is proper if and only if the original morphism is proper. Thus you are reduced to the case when the domain is quasi-projective.
Since the domain is quasi-projective it admits a (dense) open immersion into a projective scheme. So consider the "diagonal" map from the quasi-projective domain into the product of the target and this projective scheme. The diagonal map is certainly a locally closed immersion. In fact the original morphism is proper if and only if this diagonal map is a closed immersion. In other words, the original map is proper if and only if the image of the diagonal map equals its closure, i.e., if and only if the boundary is empty.
But this can easily be checked with valuations / curves of the type you are discussing. If you prefer valuations, choose any irreducible component of the image of the diagonal map whose closure intersects the boundary. Next normalize the closure of that component, and form the inverse image of the boundary in that normalization. Next blow up this inverse image to produce an exceptional Cartier divisor. Finally, take the stalk of the blowing up at any generic point of the exceptional divisor to get a DVR of the type you want.
If you prefer curves, do the same normalization and blowing up. You can arrange that everything is still quasi-projective. Now intersect with generic hyperplanes to produce an integral curve which intersects both the boundary and the strict transform of whatever dense open set you originally specified.
Best Answer
Yes.
I claim that, for any $K'$ point of $X$, if this point extends both to an $R'$ point and an $K$ point, then it extends to a $R$ point. This obviously proves the result.
Let $x'$ be the closed point of $\mathrm{Spec}(R')$. Let $\mathrm{Spec}(A)$ be an affine neighborhood of the image of $x'$. Then the $R'$ point must factor through $\mathrm{Spec}(A)$. The $K$ point also factors through $\mathrm{Spec}(A)$, as it has the same image as the $K'$ point.
We can now transform the question into algebra. The algebraic statement is that we have a map $A \to K'$ which factors through both $K$ and $R'$. But then the image of this map must lie in $K \cap R' = R$. So the map factors through $\mathrm{Spec}(R)$, and our $K$ point extends to an $R$ point.