The following statement comes from the book "Abelian varieties" by Mumford, at the very beginning of chapter 10. All varieties/schemes are defined over a field $k$.
Let $X$ be a complete variety, $Y$ any scheme and $\mathcal L$ a line bundle on $X\times Y$. Then there exists a unique closed subschemes $Y_1 \hookrightarrow Y$ such that the restriction of $\mathcal L$ to $X\times Y_1$ is isomorphic to the pullback of a line bundle $\mathcal M$ on $Y_1$ (via the projection morphism) ; and such that $Y_1$ is maximal with respect to this property.
This closed subscheme $Y_1$ is called the maximal closed subscheme of $Y$ over which $\mathcal L$ is trivial.
Now, this may be a silly question, but to my understanding we usually call a line bundle trivial when it is isomorphic to the structure sheaf of the scheme. With this in mind, why wouldn't we require the condition "the restriction of $\mathcal L$ to $X\times Y_1$ is isomorphic to $\mathcal O_{X\times Y_1}$" instead ? It doesn't seem equivalent as the line bundle $\mathcal M$ may not be trivial. Is there a specific reason for this ?
Best Answer
$\newcommand{\L}{\mathcal L} \newcommand{\M}{\mathcal M} $I believe that in this context you are supposed to think of $\L$ as a family of line bundles on $X$ parametrized by $Y$. For every $k$-point $i:y\to Y$ you can take the pullback $\widetilde i:X\to X\times Y$, and obtain a line bundle $\widetilde i^*\L$ on $X$. The proposition is stating that $Y_1$ is such that whenever $y\in Y_1$, then $\widetilde i^*\L\cong \mathcal O_X$.
If $Y$ is reduced, then being trivial on every fiber is equivalent to being a pullback from a line bundle on $Y$. Let us draw this diagram: $$\require{AMScd} \begin{CD} X @>{\widetilde i}>> X\times Y\\ @V{p}VV @V{\pi}VV \\ y @>{i}>> Y. \end{CD}$$ First, if $\L = \pi^*\M$ for some line bundle $\M$, then $\widetilde i^*\L = p^*i^*\M$, which is trivial, since it's pulled back from a point (this implication holds for any $Y$).
To prove the converse we can use this proposition you are asking about in Mumford's book. Suppose $\widetilde i^* Y$ is trivial for every $y$. By the proposition, there is a maximal closed subscheme $Y_1$ such that $\L|_{X\times Y_1}$ is a pullback of some line bundle $\M$ on $X\times Y_1$. Every point of $Y$ is a (non-maximal) closed subscheme with this property, so $Y_1$ must contain all the points of $Y$, and since $Y$ is reduced, $Y_1=Y$.
If you interpret the statement as $\L$ being trivial on $X\times Y_1$, then it's false. In the simplest example, take $X$ to be a point, and let $Y$ be such that it has a nontrivial line bundle $\L$ (which we can think of as a line bundle on $X\times Y$), for example $Y=\mathbb P^1$. Then over every point of $Y$, $\L$ is trivial, yet globally it is not, so there cannot exist such a biggest closed subscheme.