Let $ p : X \to S $ and $ q : Y \to S $ be the structure maps. We can replace $ S $ with any open affine $ W $ around $ s $, $ X $ with $ p ^{-1} ( W ) $ and $ Y $ with $ q ^ { - 1 } ( W ) $ while the hypothesis still holds. Henceforth, assume that $ S $ is the spectrum of a Noetherian ring $ R $ and $ s $ is the prime $ \mathfrak{p} $ in $ R $. Set $ L = R \setminus \mathfrak{p} $.
Claim 1. The scheme $ X \times O _{ S,s }$ is topologically a subspace of $ X $ that is the union of all the fibers $ f^{-1}(t) = X_{t} = X \times _ { S } k(t) $ where $ t $ is a generization of $ s $ i.e. $ s \in \overline { \left \{ t \right \} } $. Moreover, any open subscheme of $ X \times O_{S,s} $ is isomorphic to $ U \times_{s} \mathcal{O}_{S,s} $ for some open subscheme $ U $ of $ X $.
Proof. Let us denote the underlying topological space of $ X \times _ { S } \mathcal{O}_{S,s} $ by $ T $.
For the first part, it suffices to show the claim when $ X $ is affine (Why?). Suppose $ X = \text{Spec } A $. Then, $ X \times _ { S } \mathcal{O}_{S,s} = \text{Spec } ( A \otimes R _ { \mathfrak{p} } ) = \text{Spec } ( L ^ { - 1 } A ) $. Since this is a localization, the space $ T $ is a subspace of $ X $. Moreover, $ T $ consists of exactly those primes $ \mathfrak{q} $ of $ A $ whose inverse image under the ring map $ R \to A $ is a prime $ \mathfrak{p} ' \subset \mathfrak{p} $ i.e. $ T $ is the pre-image of those points in $ S $ which are generizations of $ \mathfrak { p } $. This proves the first half of the claim.
Suppose now that $ V $ is an open subscheme of $ X \times _{S } \mathcal{O}_{S,s} $. Then, from the proof above of the first part, $ V $ as a topological susbspace is equal to $ U \cap T $ for some open subset $ U $ of $ X $. Now, $ U $ has a unique open sub-scheme structure inherited from $ X $ and the scheme $ U \times _ { S } \mathcal{O}_{S,s} $, which is an open subscheme of $ X \times _ { S } \mathcal{O}_{S,s} $, has the same underlying topological space as $ V $. Since the open subscheme structures are unique, we must have $ V \cong U \times _ { S} \mathcal{O}_{S,s} $. $ \square $
Henceforth, we shall call the scheme $ X \times _ { S } \mathcal{O} _ { S, s} $ the thick fiber of $ X $ at $ s $ and denote it by $ ^ t X _ { s } $. So, we are given a map $$ \varphi :\ ^{t}{X} _ { s} \to \ ^ t Y _ { s } . $$
and the problem is to "expand this map" around the subspaces $ ^t{X} $, $ ^{t}Y $ to $ p^{-1}(U) $, $ q^{-1}(U) $ for some open affine $ U \ni s $ in $ S $.
Since $ X $, $ Y $ are finite type $ R $-schemes, both $ X $ and $ Y $ are quasi-compact. Let $ U = \text{Spec } A $ be an aribrary open affine subset of $ X $ and $ V _{i} = \text{Spec } A _{ i } $, $ 1 \leq i \leq n $ be a finite open affine cover of $ Y $, and $ V_{i} ' = \text{Spec } A_{i} \otimes R_{\mathfrak{p}} $ be the corresponding covering of the thick fiber of $ Y $ at $ s $. We can cover $ U \cap \varphi ^ { - 1 } ( V_{i} ' ) $ for each $ i = 1, \ldots, n $ by finitely many distinguished open subsets of $ \text{Spec } L^{-1} A = U \times _ { S } \mathcal{O}_{S,s} $, say altogether by $$ U_{j} ' = \text{Spec } A_{\mathfrak{p}, g_{j}} \text{ for } 1 \leq j \leq N , \quad g_{j} \in A _ { \mathfrak{p} } $$
so that each of these open affines lands inside $ V_ { i } ' $ for some $ i $. Then, since $ D(g_{j}) $ cover $ \text{Spec } L ^ { -1 } A
$, $ g_{j} $ generate the unit ideal in $ A _ { \mathfrak{p} } $. Suppose that $$ g_ { j } = \frac{x_{j } } { y_{ j } } $$
where $ x_{j} \in A $, $ y_{j} \in L $. Then, from the remark just made, there are $ \frac { a _{j} } { z_{j} } \in L ^ { - 1 } A $ where $ a_{j} \in A $, $ z_{j} \in L $ for $ j = 1 , \ldots, N $ such that $$ \sum_{i} \frac{ a _{i} } { z_{i} } \cdot \frac{x_{i} } { y_{i} } = 1 , $$
which, after clearing all denominators, implies that the ideal generated by $ x_{1} , \ldots , x_{N} $ in $ A $ is generated by the image of an element $ \alpha \in \eta ( L ) $, where $ \eta : R \to A $ is the ring map. If we set $ U_{j} = \text{Spec } A_{x_{j} } $, then we cannot say that $ U_{j} $ necessarily make a cover of $ U $. If however one replaces $$ R \rightsquigarrow R_{ \alpha } , $$
(and $ X $, $ Y $, $ V_{i} $, $ U $ by appropriate open subschemes)
to begin with, $ U_{j} $ can now be assumed to cover $ U $ (since $ x_{i} $ now generate the unit ideal in $ A $).
Since $ X $ is quasi-compact, we can do the same argument for each open subset in a finite cover of $ X $. We thus end up with an affine cover $ \text{Spec } A_{j} $ of $ X $ for $ 1 \leq j \leq m $, a cover $ \text{Spec } B_{i} $ of $ Y $ for $ 1 \leq i \leq n $, and a finite collection of ring maps $ B_{i} \otimes R _ { \mathfrak { p } } \to A_{j} \otimes R _ { \mathfrak{p} } $ for pairs $ (i,j) $ in some set $ I \subset \left {1, \ldots, n \right \} \times \left\{ 1, m \right \} $ that agree on intersections.
Claim 2. Let $ A $ and $ B $ be $ R $-algebras where $ R $ is a Noetherian ring and let $ \mathfrak{p} $ a prime of $ R $. Suppose that we have a ring map $ \phi : B \otimes R_{ \mathfrak{ p } } \to A \otimes R _ { \mathfrak{p} } $. If $ B $ is finitely generated, there exists a $ r \in R \setminus \mathfrak{p} $ such that $ \phi $ is obtained from a map $ \psi : B _ { r } \to A_{ r } $ by localizing to $ R \setminus \mathfrak{p} $. Moreover, if $ r' $ and $ \psi ' : B _ { r '} \to A_{r' } $ are any other such pair, then the maps so defined from $ B_{rr' } \to A_{rr' } $ by localizing $ \psi , \psi ' $ are the same.
Proof. Suppose that $ B = R[t_{1}, \ldots, t_{n} ] / ( b_{1} , \ldots, b _ { m } ) $ where $ b_{1}, \ldots, b_{m} \in R [t_{1}, \ldots, t_{n} ] $. Consider the induced map $ \chi : R _ { \mathfrak{p} } [ t_{1} , \ldots, t_{n} ] \to A_{ \mathfrak{p} } $. Suppose that $$ t_{i} \mapsto \frac{a_{i}}{s_{i}} \quad \quad \quad \text{for } i = 1, \ldots, n $$
and let $ s = s_{1} s_{2} \ldots s_{n} $. Define a map $ \chi ' : R_{s} [ t_{1} , \ldots, t_{n } ] \to A_{ s } $ by $$ t_{i} \mapsto
\frac{a_{i} s_{1} \cdots s_{i-1} s_{i+1} \cdots s_{n} } { s }
\quad \quad \quad \text{ for } i = 1 , \ldots, n $$
and suppose that under $ \chi ' $, $$ b_{j} \mapsto \frac{ a_{j} ' } { s_{j} ' } \text{ for } j = 1, \ldots, m . $$
It is clear that $ \chi $ is obtained from $ \chi ' $ after localizing to $ R \setminus \mathfrak{p} $. Thus, each $ \frac{ a_{j} ' } { s _{j} ' } $ becomes a zero in $ A _ { \mathfrak { p } } $, which means that there are $ r_{j} \in R \setminus \mathfrak{p} $ such that $ r _{j} \cdot a_{j} ' = 0 $. Thus, taking $ r = s \cdot r _{1} r _{2} \cdots r _ {m} $, we see that the induced map $$ R_{r} [ t_{1} , \ldots , t_{n} ] \to A_{r} $$
sends $ b_{j} $ to zero, and thus further induces a map $$ \psi : B _ { r } \to A_{ r } . $$
This map $ \psi $ clearly localizes to $ \phi $ and is the desired map. The fact that any two such maps are compatible is straightforward. $ \square $
In light of Claim 2 above, we can find finitely many $ r_{ij} $ for $ (i, j ) \in I $ such that the maps $ B_{j} \otimes R_{ \mathfrak{p} } \to A_{i} \otimes R _ { \mathfrak { p} } $ are localizations of some maps $$ B _ { j } \otimes R _ { r _ { ij } } \to A_{i} \otimes R _ { r _ { i j } } $$ which agree on common open subsets. By taking the product of $ r_ { i j } $ to be $ r $, we obtain a common neighbourhood $ W = D(r) $ in $ \text{Spec } R $ and compatible maps $ B_ { j } \otimes R_{r} \to A_{i} \otimes R_{r} $. This amounts to giving a map $$ p^{-1} ( W ) \to q ^ {- 1} ( W ) $$
and which is the desired claim.
Remark 1. We only needed the map $ X \to S $ to be quasi-compact instead of finite type in the proof.
It is interesting to note that no hypothesis on $ X $ was needed in Extending a morphism of schemes, but is crucial here for the result to hold. The necessity of some hypothesis on $ X $ was pointed out to me Arnav Tripathy and Koji Shizimu and an explicit counterexample given by the latter is reproduced below.
Take $ S = \text{Spec } \mathbb{Z} $, $ s $ the generic point of $ S $, $ Y = \text{Spec } \mathbb{Z} [t] $ and $ X = \bigsqcup _ { i \in \mathbb{N} } X_{i} $ where $ X_{i} = \text{Spec } \mathbb{Z} [ t] $. Then, $ X \times
\mathcal{ O } _{S,s} = \text{Spec } \mathbb{Q} [t ] $ and $ X \times _ { S } \mathcal{O} _{ S, s } = \bigsqcup_{ i \in \mathbb{N} } \mathbb{Q} [ t ] $. We can then define a map $ \varphi : X \times _ { S } \mathcal{O}_{S,s} \to Y \times \mathcal{O}_{S,s} $ by defining the map on the $ i $-th copy of $ \mathbb{Q} [ t] $ to $ \mathbb{Q} [ t] $ which sends $ t $ to $ t / i $. Since the open subsets of $ \text{Spec } \mathbb{Z} $ are of the form $ \text{Spec } \mathbb{Z} [ 1 / N ] $, it is impossible to find an open subset $ U $ of $ S $ such that $ \varphi $ is obtained from some $ X \times U \to Y \times U $, as there are infinitely many denominators involved.
Remark 2. The claim that open subsets of $ X \times \mathcal{O}_{S,s} $ are of the form $ U \times \mathcal{O}_{S,s} $ isn't true if $ \mathcal{O} _ { S , s } $ is replaced by an arbitrary $ S $-scheme $ Z $. See the counterexample here.
Remark 3. One may wonder that if any open subset of $ X \times \mathcal{O}_{S,s} $ is of the form $ U \times _{ S} \mathcal{O}_{S,s} $ for some open subset $ U $ of $ X $, how does one write $ X \times_{S} W $ in the said form where $ W $ is an open subset of $ \mathcal{O}_{S,s} $? This may seem counter-intuitive as first, but we can proceed as follows.
Since $ \text{Spec } \mathcal{O}_{S,s} $ is a subspace of $ S $ (it is, in fact, the intersection of all open subsets of $ S $ containing $ s $), any open subset $ W $ of $ \text{Spec } \mathcal{O}_{S,s} $ is obtained by intersecting some open subset $ V $ of $ S $ with this subspace. Let $ U = p ^ { - 1 } ( V ) $. Then, $ X \times _ { S } W = U \times _ { S } \mathcal{O}_{S,s} $.
In a normal scheme, irreducible components are connected components: no two irreducible components may meet, as otherwise the local ring at a point in their intersection would not be a domain, contradicting the definition that all local rings in a normal scheme are integrally closed domains. Therefore it makes sense to analyze component-by-component for Weil divisors on normal schemes, and one may just apply the relevant results on each component and conclude from there. In particular, Weil divisors are perfectly well defined on an arbitrary noetherian normal scheme.
As to the comparison between Cartier divisors (line bundles) and Weil divisors (codimension one cycles), the following material is relatively standard. We'll roughly follow Chapter 14 of Vakil's FOAG.
Definition. Let $X$ be a noetherian scheme. Define a map $\operatorname{div}$ from the collection of line bundles $\mathcal{L}$ on $X$ with a rational section $s$ not vanishing on any irreducible component of $X$ to Weil divisors on $X$ as follows:
$$\operatorname{div}(s) = \sum_{Y} \operatorname{val}_Y(s)\cdot [Y] $$
where $Y$ ranges over the codimension one irreducible subschemes of $X$, and $\operatorname{val}_Y$ represents the natural valuation.
This gives a map from the group under tensor product of isomorphism classes of line bundles with rational section to the group of Weil divisors.
Proposition. (Vakil 14.2.1) Let $X$ be a noetherian normal scheme. The map $\operatorname{div}(s):Pic(X)\to Cl(X)$ is injective.
This means for any normal noetherian scheme, any Cartier divisor gives a Weil divisor. We see that the reverse is not always true: the canonical example is the divisor $D$ given by the line $V(x,z)$ inside the cone $V(xy-z^2)\subset \Bbb A^3$ (Vakil exercise 14.2.H). $D$ is not Cartier, because the divisor vanishes to order 2 on the set-theoretic support of the divisor. On the other hand, $2D$ is a Cartier divisor, and this is essentially the only way in which things can go wrong.
Definition. A scheme is called factorial (or locally factorial) if every local ring is a unique factorization domain.
We note that as UFDs are normal domains, this immediately implies that every factorial scheme is in fact normal.
Proposition (Vakil 14.2.10). Let $X$ be a noetherian factorial scheme. Then for any Weil divisor $D$, the sheaf $\mathcal{O}(D)$ is a line bundle, and the map $Pic(X)\to Cl(X)$ is an isomorphism.
In particular, this means on a factorial noetherian scheme, the notions of Weil divisor and Cartier divisor are the same.
Best Answer
First, you should convince yourself that the question whether a given morphism $\operatorname{Spec} \mathcal{O}_{X,x} \to Y$ extends to an open neighborhood of $x$ comes down to the following question in commutative algebra:
Note that there is no reason to hope that such a factorization always exists. Indeed, the answer to the above question might be "no".
But what if $B$ is a finitely generated $R$-algebra with $R$ a Noetherian ring, $A$ is an $R$-algebra and the given morphism is a morphism of $R$-algebras? This is the case you have to deal with in solving your exercise (I'll assume you are able to work out why - if you have trouble, just drop a comment). Then the above question is guaranteed to have a positive answer. Let's prove this. Write $B = R[T_1,\dotsc,T_n] / (g_1,\dotsc,g_m)$, denote the composite of the canonical projection $R[T_1,\dotsc,T_n] \twoheadrightarrow B$ with the given morphism $B \to A_\mathfrak{p}$ by $\varphi$ and write
$$\varphi(T_i) = \frac{a_i}{f_i}, \quad i=1,\dotsc,n \, ,$$ $$\varphi(g_j) = \frac{a_j'}{h_j}, \quad j=1,\dotsc,m \, .$$
Now, for each $j \in \lbrace 1,\dotsc,m \rbrace$, choose some $f_j' \in A \setminus \mathfrak{p}$ such that $f_j' a_j' = 0$ in $A$ (which exists since $\varphi(g_j) = 0 \in A_\mathfrak{p}$). I now claim that $f = \prod_{i=1}^n f_i \prod_{j=1}^m f_j'$ has the desired property, i.e. that the given homomorphism factors through $A_f \to A_\mathfrak{p}$. To see why, we first make use of the fact that, by construction, $f$ is a common denominator of all the $\varphi(T_i)$ and note that
$$ T_i \mapsto \frac{a_i \prod_{k \neq i} f_k \prod_{j=1}^m f_j'}{f}, \quad i=1,\dotsc,n$$
defines a morphism of $R$-algebras $R[T_1,\dotsc,T_n] \to A_f$ whose composite with $A_f \to A_\mathfrak{p}$ agrees with $\varphi$. In addition, we have, by construction, $a_j' f = 0$ for all $j \in \lbrace 1,\dotsc,m \rbrace$; thus, the morphism just defined maps each $g_j$ to $0$ and, as a consequence, factors over $B$.