For any subset $|Y| \subseteq |X|$, there exists a monomorphism $\iota \colon Y \hookrightarrow X$ supported on $Y$. However, these do not have the property that any map landing in $|Y|$ factors through $Y$.
In fact, I will show (using results from the Samuel seminar on epimorphisms of rings [Sam], [Laz]) that in the example $\mathbb A^2 \to \mathbb A^2$ given by $(x,y) \mapsto (x,xy)$ that you give, there is no minimal monomorphism it factors through.
Lemma 1. For any subset $|Y| \subseteq |X|$, there exists a monomorphism $\iota \colon Y \hookrightarrow X$ supported on $Y$.
Proof. Indeed, let $Y = \coprod_{y \in |Y|} \operatorname{Spec} \kappa(y)$. Then the natural map $\iota \colon Y \to X$ is a monomorphism, because $Y \times_X Y$ is just $Y$. (See [ML, Exc. III.4.4] for this criterion for monomorphism.) $\square$
Now we focus on the example $\mathbb A^2 \to \mathbb A^2$ given by $(x,y) \mapsto (x,xy)$. Write $f \colon Z \to X$ for this map, and note that $f$ is an isomorphism over $D(x) \subseteq X$. Assume $\iota \colon Y \to X$ is a minimal immersion such that there exists a factorisation of $f$ as
$$Z \stackrel g\to Y \stackrel \iota \to X.$$
For any irreducible scheme $S$, write $\eta_S$ for its generic point. We denote the origin of $\mathbb A^2$ by $0$.
Lemma 2. We must have $|Y| = |\!\operatorname{im}(f)|$ or $|Y| = |\!\operatorname{im}(f)| \cup \{\eta_{V(x)}\}$, and $Y$ is integral.
Proof. Clearly $|Y|$ contains $|\!\operatorname{im}(f)|$. If this inclusion were strict, then $|Y|$ contains some point $y$ that is not in the image of $f$. If $y$ is closed in $X$, then the open immersion $Y \setminus \{y\} \to Y$ gives a strictly smaller monomorphism that $f$ factors through, contradicting the choice of $Y$. Thus, $y$ has to be the generic point of $V(x)$. This proves the first statement.
For the second, note that the scheme-theoretic image $\operatorname{im}(g)$ of $g$ is $Y$. Indeed, if it weren't, then replacing $Y$ by $\operatorname{im}(g)$ would give a smaller monomorphism factoring $f$, contradicting minimality of $Y$. But the scheme-theoretic image of an integral scheme is integral, proving the second statement. $\square$
We now apply the following two results from the Samuel seminar on epimorphisms of rings [Sam]:
Theorem. Let $\iota \colon Y \to X$ be a quasi-compact birational monomorphism of integral schemes, with $X$ normal and locally Noetherian. Then $\iota$ is flat.
Proof. See [Sam, Lec. 7, Cor. 3.6]. $\square$
If $U \subseteq Y$ is an affine open neighbourhood of $0$ and $R = \Gamma(U,\mathcal O_U)$, then we get a flat epimorphism $\phi \colon k[x,y] \to R$ of $k$-algebras (not necessarily of finite type).
Theorem. Let $f \colon A \to B$ be a flat epimorphism of rings, and assume $A$ is normal and $\operatorname{Cl}(A)$ is torsion. Then $f$ is a localisation, i.e. $B = S^{-1}A$ for $S \subseteq A$ a multiplicative subset.
Proof. See [Laz, Prop. IV.4.5]. $\square$
Thus, $R = S^{-1}k[x,y]$ for some multiplicative set $S \subseteq k[x,y]$. This implies that $V = X \setminus U$ is a union of (possibly infinitely many) divisors. Moreover, $V \cap D(x) \subseteq D(x)$ has finitely many components since $D(x)$ is Noetherian (and $\iota$ is an isomorphism over $D(x)$). But then $V \cap V(x)$ is either finite or all of $V(x)$. This contradicts the fact that $U \cap V(x)$ equals $\{0\}$ or $\{0,\eta_{V(x)}\}$ (depending whether $\eta_{V(x)} \in Y$). $\square$
References.
[Laz] D. Lazard, Autour de la platitude, Bull. Soc. Math. Fr. 97, 81-128 (1969). ZBL0174.33301.
[ML] S. Mac Lane, Categories for the working mathematician. Graduate Texts in Mathematics 5. New York-Heidelberg-Berlin: Springer-Verlag (1971). ZBL0232.18001.
[Sam] P. Samuel et al., Séminaire d’algèbre commutative (1967/68): Les épimorphismes d’anneaux. Paris: École Normale Supérieure de Jeunes Filles (1968). ZBL0159.00101.
Normality
For an integral scheme, being normal (integrally closed in ones own fraction field) satisfies this property. Indeed, suppose that $a, b \in A = \Gamma(X, O_X)$ are such that $a/b$ satisfy some polynomial $p(x) \in A[x]$. Then $a|_U, b|_U$ satisfy the same polynomial after restriction to each (affine) set $U \subseteq X$.
Of course this shows up in many applications of things like Stein factorization.
Semi-normality
A reduced ring $R$ is seminormal if for any finite extension $R \subseteq S$ satisfying the following two properties is an isomorphism.
- The induced map on $\text{Spec}$'s is an isomorphism
- The induced residue field extensions $k(r) \subseteq k(s)$ are isomorphisms for all $s \in \text{Spec } S$ mapping to $r \in \text{Spec } R$.
The typical example of a seminormal ring is a node, the cusp $k[x^2,x^3]$ is not seminormal
Equivalently, $R$ is seminormal if and only if for any $a/b$ in the total ring of fractions of $R$, one has that if $(a/b)^2, (a/b)^3 \in R$ then $(a/b) \in R$ (see a paper by Swan, he might be assuming finitely many minimal primes, I forget the details). It follows similarly that seminormality satisfies this property.
Weak normality
Weak normality is similar to semi-normality. A reduced ring is called weakly normal if for any finite birational extension $R \subseteq S$ satisfying the following properties is an isomorphism:
- The induced map on $\text{Spec}$'s is an isomorphism
- The induced residue field extensions $k(r) \subseteq k(s)$ are purely inseparable for all $s \in \text{Spec } S$ mapping to $r \in \text{Spec } R$.
This can also be phrased as requiring that every birational universal homeomorphism is an isomorphism.
I do NOT know if weakly normal rings satisfy the sort of property asked for. I do not think it is in the literature (but perhaps I am wrong). I remember I convinced myself that they did not several years ago, but never wrote down an example carefully.
Best Answer
This already fails in the affine case. For example, if $Y = \mathbf A^1$ and $X = \mathbf A^1 \setminus \{0\}$, then $Y$ is the scheme-theoretic image of $X \hookrightarrow Y$. But if $Z$ is the line with two origins, then the two inclusions $Y \to Z$ agree on $X$ but are not identical.
There is a positive result when $f$ has closed image, for then $X \to \operatorname{im}(f)$ is surjective (see also the introduction to this question). For schemes of finite type over a Jacobson ring (e.g. $\mathbf Z$ or a field $k$), this is the only way $X \to \operatorname{im}(f)$ can be an epimorphism:
Lemma. Let $f \colon X \to Y$ be an epimorphism of schemes, and let $y \in Y$ be a closed point. Then there exists $x \in X$ with $f(x) = y$. In particular, if $Y$ is Jacobson and $f$ of finite type, then $f$ is surjective.
Proof. For the first statement, consider the open $U = Y\setminus\{y\}$ and let $Z$ be two copies of $Y$ glued along $U$. The two natural inclusions $Y \to Z$ differ, so their restrictions to $X$ differ, showing that $y$ is in the (set-theoretic) image of $f$. The second statement follows from Chevalley's theorem. $\square$
However, there exist epimorphisms between non-Jacobson schemes that are not surjective:
Example. Let $A$ be a local domain of dimension $\geq 2$; for example $A = k[x,y]_{(x,y)}$. Let $A \subseteq B$ be a local homomorphism to a discrete valuation ring $B$; for example by blowing up the closed point of $A$ and localising at a generic point of the exceptional fibre. Let $X = \operatorname{Spec} B$ and $Y = \operatorname{Spec} A$. Then $f \colon X \to Y$ is not surjective (it only hits the generic point $\eta$ and the closed point $s$), but it is an epimorphism. Indeed, if $g,h \colon Y \rightrightarrows Z$ agree on $X$, then $g(s) = h(s)$. If $U \subseteq Z$ is an affine open neighbourhood of $g(s)$, then $g$ and $h$ both land in $U$, since the only open subset of $Y$ containing $s$ is $Y$ itself. The result then follows since $A \subseteq B$ is a monomorphism of rings. $\square$