Let $X$ be an integral scheme and let $Y$ be a non-empty open subscheme of $X$. $X$ is irreducible, so $Y$ is dense in $X$. I claim the induced homomorphism $i^\flat : \mathscr{O}_X \to i_* \mathscr{O}_Y$ is monic but not necessarily epic. Indeed, the claim is local on $X$ and $Y$, so we may take $X$ to be affine and $Y$ to be a distinguished open subscheme of $X$. But then all we have is a localisation of an integral domain, and this is always injective but not necessarily surjective.
The point is that there isn't one notion of subscheme which gives rise to open and closed subschemes; rather, there are two.
Open subschemes are a special case of the following construction: if $X$ is a locally ringed space and $Y$ is any subset of $X$, we can make $Y$ into a locally ringed space by pulling back $\mathscr{O}_X$ along the inclusion $i : Y \hookrightarrow X$. Unfortunately, there's no guarantee that $(Y, i^{-1} \mathscr{O}_X)$ is a scheme even if $(X, \mathscr{O}_X)$ is.
On the other hand, the definition of a closed subscheme depends a lot on the scheme structure. If $Y$ is a closed subset of $X$, that means $Y \cap U$ is closed in $U$ for every open subset $U$ – but we know that closed subsets of $\operatorname{Spec} A$ are homeomorphic to $\operatorname{Spec} A / \mathfrak{a}$ for some suitable ideal $\mathfrak{a}$, and in essence the structure of $Y$ as a closed subscheme of $X$ is defined so that we have an exact sequence
$$0 \longrightarrow i^{-1} \mathscr{I}_{Y \mid X} \longrightarrow i^{-1} \mathscr{O}_X \longrightarrow \mathscr{O}_Y \longrightarrow 0$$
or equivalently, so that we have the "fundamental exact sequence":
$$0 \longrightarrow \mathscr{I}_{Y \mid X} \longrightarrow \mathscr{O}_X \longrightarrow i_* \mathscr{O}_Y \longrightarrow 0$$
But the story for varieties is more subtle. For the purposes of this discussion, I mean "variety" in the sense of a reduced scheme of finite type over an algebraically closed field $k$. Because varieties have enough closed points, the structure sheaf of a variety $X$ is isomorphic to a subsheaf of the sheaf of continuous functions $X(k) \to \mathbb{A}^1(k)$. (Henceforth, I will pretend non-closed points don't exist.) Thus, there is a canonical way of restricting regular functions on (any open subset of) $X$ to any (not necessarily open or closed!) subset $Y$ of $X$. If $Y$ is open, this recovers the open subscheme structure, and if $Y$ is closed, this recovers the closed subscheme structure.
Let's look at this more closely. We define $\mathscr{I}_{Y \mid X}$ to be the subsheaf of $\mathscr{O}_X$ consisting of those regular functions which vanish on $Y$, i.e.
$$\mathscr{I}_{Y \mid X} (U) = \{ f \in \mathscr{O}_X : \forall y \in Y . \, f (y) = 0 \}$$
and we define, for each open subset $V$ of $Y$,
$$\mathscr{O}_Y (V) = \varinjlim_{U \supseteq V} \mathscr{O}_X (U) / \mathscr{I}_{Y \mid X} (U)$$
This is a sheaf because $Y$ is quasicompact. (Every subset of $X$ is quasicompact!) If $Y$ is open, then $V$ is open in $X$, so we are taking the direct limit over a directed system with a terminal object – hence $\mathscr{O}_Y (V) \cong \mathscr{O}_X (V) / \mathscr{I}_{Y \mid X} (V) \cong \mathscr{O}_X (V)$ in this case.
For $Y$ closed something weird happens as well. Let $U$ be an open affine subset of $X$. Then, $V = U \cap Y$ is a closed subset of $U$ and an open subset of $Y$. Suppose $f \in \mathscr{O}_X (U)$ does not vanish on $V$. Then, the Nullstellensatz implies $f$ is already invertible in $\mathscr{O}_X (U) / \mathscr{I}_{Y \mid X} (U)$ – and this implies that the directed system is constant! In particular, we get $i_* \mathscr{O}_Y \cong \mathscr{O}_X / \mathscr{I}_{Y \mid X}$.
In general, however, we don't get anything nice. There is a natural left exact sequence of groups
$$0 \longrightarrow \mathscr{I}_{Y \mid X} (U) \longrightarrow \mathscr{O}_X (U) \longrightarrow \mathscr{O}_Y (U \cap Y)$$
and therefore a left exact sequence of sheaves on $X$:
$$0 \longrightarrow \mathscr{I}_{Y \mid X} \longrightarrow \mathscr{O}_X \longrightarrow i_* \mathscr{O}_Y$$
We have just seen that this extends to a short exact sequence when $Y$ is closed. When $Y$ is open and $X$ is irreducible, the homomorphism $\mathscr{O}_X \to i_* \mathscr{O}_Y$ is monic but in general not epic. (Consider a point $x \in X \setminus Y$: the stalk of $i_* \mathscr{O}_Y$ at $x$ gives the fraction field of the local ring $\mathscr{O}_{X, x}$.)
Yes, that's a correct definition. Yours (1.) is also equivalent to 2. below.
- $f(X)$ is open (closed) and $f$ is a homeomorphism on its image
- $f$ is open (closed) and a homeomorphism on its image
If we then define an immersion to be a homeomorphism on its image, then an open (closed) immersion really is an immersion that is open (closed).
Note: it is also called an embedding, which is safer to use than immersion, because it is closer to the terminology used in differential geometry.
Best Answer
No, this is not correct - if $y$ is not in the image of $f$, then $(f_*\mathcal{O}_X)_y$ is zero. This isn't a problem for the case of a closed immersion, but it is for an open immersion.