Let $ p : X \to S $ and $ q : Y \to S $ be the structure maps. We can replace $ S $ with any open affine $ W $ around $ s $, $ X $ with $ p ^{-1} ( W ) $ and $ Y $ with $ q ^ { - 1 } ( W ) $ while the hypothesis still holds. Henceforth, assume that $ S $ is the spectrum of a Noetherian ring $ R $ and $ s $ is the prime $ \mathfrak{p} $ in $ R $. Set $ L = R \setminus \mathfrak{p} $.
Claim 1. The scheme $ X \times O _{ S,s }$ is topologically a subspace of $ X $ that is the union of all the fibers $ f^{-1}(t) = X_{t} = X \times _ { S } k(t) $ where $ t $ is a generization of $ s $ i.e. $ s \in \overline { \left \{ t \right \} } $. Moreover, any open subscheme of $ X \times O_{S,s} $ is isomorphic to $ U \times_{s} \mathcal{O}_{S,s} $ for some open subscheme $ U $ of $ X $.
Proof. Let us denote the underlying topological space of $ X \times _ { S } \mathcal{O}_{S,s} $ by $ T $.
For the first part, it suffices to show the claim when $ X $ is affine (Why?). Suppose $ X = \text{Spec } A $. Then, $ X \times _ { S } \mathcal{O}_{S,s} = \text{Spec } ( A \otimes R _ { \mathfrak{p} } ) = \text{Spec } ( L ^ { - 1 } A ) $. Since this is a localization, the space $ T $ is a subspace of $ X $. Moreover, $ T $ consists of exactly those primes $ \mathfrak{q} $ of $ A $ whose inverse image under the ring map $ R \to A $ is a prime $ \mathfrak{p} ' \subset \mathfrak{p} $ i.e. $ T $ is the pre-image of those points in $ S $ which are generizations of $ \mathfrak { p } $. This proves the first half of the claim.
Suppose now that $ V $ is an open subscheme of $ X \times _{S } \mathcal{O}_{S,s} $. Then, from the proof above of the first part, $ V $ as a topological susbspace is equal to $ U \cap T $ for some open subset $ U $ of $ X $. Now, $ U $ has a unique open sub-scheme structure inherited from $ X $ and the scheme $ U \times _ { S } \mathcal{O}_{S,s} $, which is an open subscheme of $ X \times _ { S } \mathcal{O}_{S,s} $, has the same underlying topological space as $ V $. Since the open subscheme structures are unique, we must have $ V \cong U \times _ { S} \mathcal{O}_{S,s} $. $ \square $
Henceforth, we shall call the scheme $ X \times _ { S } \mathcal{O} _ { S, s} $ the thick fiber of $ X $ at $ s $ and denote it by $ ^ t X _ { s } $. So, we are given a map $$ \varphi :\ ^{t}{X} _ { s} \to \ ^ t Y _ { s } . $$
and the problem is to "expand this map" around the subspaces $ ^t{X} $, $ ^{t}Y $ to $ p^{-1}(U) $, $ q^{-1}(U) $ for some open affine $ U \ni s $ in $ S $.
Since $ X $, $ Y $ are finite type $ R $-schemes, both $ X $ and $ Y $ are quasi-compact. Let $ U = \text{Spec } A $ be an aribrary open affine subset of $ X $ and $ V _{i} = \text{Spec } A _{ i } $, $ 1 \leq i \leq n $ be a finite open affine cover of $ Y $, and $ V_{i} ' = \text{Spec } A_{i} \otimes R_{\mathfrak{p}} $ be the corresponding covering of the thick fiber of $ Y $ at $ s $. We can cover $ U \cap \varphi ^ { - 1 } ( V_{i} ' ) $ for each $ i = 1, \ldots, n $ by finitely many distinguished open subsets of $ \text{Spec } L^{-1} A = U \times _ { S } \mathcal{O}_{S,s} $, say altogether by $$ U_{j} ' = \text{Spec } A_{\mathfrak{p}, g_{j}} \text{ for } 1 \leq j \leq N , \quad g_{j} \in A _ { \mathfrak{p} } $$
so that each of these open affines lands inside $ V_ { i } ' $ for some $ i $. Then, since $ D(g_{j}) $ cover $ \text{Spec } L ^ { -1 } A
$, $ g_{j} $ generate the unit ideal in $ A _ { \mathfrak{p} } $. Suppose that $$ g_ { j } = \frac{x_{j } } { y_{ j } } $$
where $ x_{j} \in A $, $ y_{j} \in L $. Then, from the remark just made, there are $ \frac { a _{j} } { z_{j} } \in L ^ { - 1 } A $ where $ a_{j} \in A $, $ z_{j} \in L $ for $ j = 1 , \ldots, N $ such that $$ \sum_{i} \frac{ a _{i} } { z_{i} } \cdot \frac{x_{i} } { y_{i} } = 1 , $$
which, after clearing all denominators, implies that the ideal generated by $ x_{1} , \ldots , x_{N} $ in $ A $ is generated by the image of an element $ \alpha \in \eta ( L ) $, where $ \eta : R \to A $ is the ring map. If we set $ U_{j} = \text{Spec } A_{x_{j} } $, then we cannot say that $ U_{j} $ necessarily make a cover of $ U $. If however one replaces $$ R \rightsquigarrow R_{ \alpha } , $$
(and $ X $, $ Y $, $ V_{i} $, $ U $ by appropriate open subschemes)
to begin with, $ U_{j} $ can now be assumed to cover $ U $ (since $ x_{i} $ now generate the unit ideal in $ A $).
Since $ X $ is quasi-compact, we can do the same argument for each open subset in a finite cover of $ X $. We thus end up with an affine cover $ \text{Spec } A_{j} $ of $ X $ for $ 1 \leq j \leq m $, a cover $ \text{Spec } B_{i} $ of $ Y $ for $ 1 \leq i \leq n $, and a finite collection of ring maps $ B_{i} \otimes R _ { \mathfrak { p } } \to A_{j} \otimes R _ { \mathfrak{p} } $ for pairs $ (i,j) $ in some set $ I \subset \left {1, \ldots, n \right \} \times \left\{ 1, m \right \} $ that agree on intersections.
Claim 2. Let $ A $ and $ B $ be $ R $-algebras where $ R $ is a Noetherian ring and let $ \mathfrak{p} $ a prime of $ R $. Suppose that we have a ring map $ \phi : B \otimes R_{ \mathfrak{ p } } \to A \otimes R _ { \mathfrak{p} } $. If $ B $ is finitely generated, there exists a $ r \in R \setminus \mathfrak{p} $ such that $ \phi $ is obtained from a map $ \psi : B _ { r } \to A_{ r } $ by localizing to $ R \setminus \mathfrak{p} $. Moreover, if $ r' $ and $ \psi ' : B _ { r '} \to A_{r' } $ are any other such pair, then the maps so defined from $ B_{rr' } \to A_{rr' } $ by localizing $ \psi , \psi ' $ are the same.
Proof. Suppose that $ B = R[t_{1}, \ldots, t_{n} ] / ( b_{1} , \ldots, b _ { m } ) $ where $ b_{1}, \ldots, b_{m} \in R [t_{1}, \ldots, t_{n} ] $. Consider the induced map $ \chi : R _ { \mathfrak{p} } [ t_{1} , \ldots, t_{n} ] \to A_{ \mathfrak{p} } $. Suppose that $$ t_{i} \mapsto \frac{a_{i}}{s_{i}} \quad \quad \quad \text{for } i = 1, \ldots, n $$
and let $ s = s_{1} s_{2} \ldots s_{n} $. Define a map $ \chi ' : R_{s} [ t_{1} , \ldots, t_{n } ] \to A_{ s } $ by $$ t_{i} \mapsto
\frac{a_{i} s_{1} \cdots s_{i-1} s_{i+1} \cdots s_{n} } { s }
\quad \quad \quad \text{ for } i = 1 , \ldots, n $$
and suppose that under $ \chi ' $, $$ b_{j} \mapsto \frac{ a_{j} ' } { s_{j} ' } \text{ for } j = 1, \ldots, m . $$
It is clear that $ \chi $ is obtained from $ \chi ' $ after localizing to $ R \setminus \mathfrak{p} $. Thus, each $ \frac{ a_{j} ' } { s _{j} ' } $ becomes a zero in $ A _ { \mathfrak { p } } $, which means that there are $ r_{j} \in R \setminus \mathfrak{p} $ such that $ r _{j} \cdot a_{j} ' = 0 $. Thus, taking $ r = s \cdot r _{1} r _{2} \cdots r _ {m} $, we see that the induced map $$ R_{r} [ t_{1} , \ldots , t_{n} ] \to A_{r} $$
sends $ b_{j} $ to zero, and thus further induces a map $$ \psi : B _ { r } \to A_{ r } . $$
This map $ \psi $ clearly localizes to $ \phi $ and is the desired map. The fact that any two such maps are compatible is straightforward. $ \square $
In light of Claim 2 above, we can find finitely many $ r_{ij} $ for $ (i, j ) \in I $ such that the maps $ B_{j} \otimes R_{ \mathfrak{p} } \to A_{i} \otimes R _ { \mathfrak { p} } $ are localizations of some maps $$ B _ { j } \otimes R _ { r _ { ij } } \to A_{i} \otimes R _ { r _ { i j } } $$ which agree on common open subsets. By taking the product of $ r_ { i j } $ to be $ r $, we obtain a common neighbourhood $ W = D(r) $ in $ \text{Spec } R $ and compatible maps $ B_ { j } \otimes R_{r} \to A_{i} \otimes R_{r} $. This amounts to giving a map $$ p^{-1} ( W ) \to q ^ {- 1} ( W ) $$
and which is the desired claim.
Remark 1. We only needed the map $ X \to S $ to be quasi-compact instead of finite type in the proof.
It is interesting to note that no hypothesis on $ X $ was needed in Extending a morphism of schemes, but is crucial here for the result to hold. The necessity of some hypothesis on $ X $ was pointed out to me Arnav Tripathy and Koji Shizimu and an explicit counterexample given by the latter is reproduced below.
Take $ S = \text{Spec } \mathbb{Z} $, $ s $ the generic point of $ S $, $ Y = \text{Spec } \mathbb{Z} [t] $ and $ X = \bigsqcup _ { i \in \mathbb{N} } X_{i} $ where $ X_{i} = \text{Spec } \mathbb{Z} [ t] $. Then, $ X \times
\mathcal{ O } _{S,s} = \text{Spec } \mathbb{Q} [t ] $ and $ X \times _ { S } \mathcal{O} _{ S, s } = \bigsqcup_{ i \in \mathbb{N} } \mathbb{Q} [ t ] $. We can then define a map $ \varphi : X \times _ { S } \mathcal{O}_{S,s} \to Y \times \mathcal{O}_{S,s} $ by defining the map on the $ i $-th copy of $ \mathbb{Q} [ t] $ to $ \mathbb{Q} [ t] $ which sends $ t $ to $ t / i $. Since the open subsets of $ \text{Spec } \mathbb{Z} $ are of the form $ \text{Spec } \mathbb{Z} [ 1 / N ] $, it is impossible to find an open subset $ U $ of $ S $ such that $ \varphi $ is obtained from some $ X \times U \to Y \times U $, as there are infinitely many denominators involved.
Remark 2. The claim that open subsets of $ X \times \mathcal{O}_{S,s} $ are of the form $ U \times \mathcal{O}_{S,s} $ isn't true if $ \mathcal{O} _ { S , s } $ is replaced by an arbitrary $ S $-scheme $ Z $. See the counterexample here.
Remark 3. One may wonder that if any open subset of $ X \times \mathcal{O}_{S,s} $ is of the form $ U \times _{ S} \mathcal{O}_{S,s} $ for some open subset $ U $ of $ X $, how does one write $ X \times_{S} W $ in the said form where $ W $ is an open subset of $ \mathcal{O}_{S,s} $? This may seem counter-intuitive as first, but we can proceed as follows.
Since $ \text{Spec } \mathcal{O}_{S,s} $ is a subspace of $ S $ (it is, in fact, the intersection of all open subsets of $ S $ containing $ s $), any open subset $ W $ of $ \text{Spec } \mathcal{O}_{S,s} $ is obtained by intersecting some open subset $ V $ of $ S $ with this subspace. Let $ U = p ^ { - 1 } ( V ) $. Then, $ X \times _ { S } W = U \times _ { S } \mathcal{O}_{S,s} $.
Best Answer
You can endow $B$ with the trivial graded structure, $B_0 = A, B_i = B \ \forall \ i > 0$. Then show that you have a graded ring morphism $\varphi \colon A[T,S]/(\widetilde{P}(T,S)) \longrightarrow A[T]/(P(T))$ given by $T \longmapsto T, S \longmapsto 1$. If the LHS is inherited the natural graded structure of $A[T,S]$ ($\widetilde{P}(T,S)$ is homogeneous), then $\varphi$ is a morphism of graded rings. By lemma 3.40 in Qing Liu, you have a morphism $\operatorname{Proj}(B) \setminus V_+(\varphi(C_+)B) \longrightarrow \operatorname{Proj}(A[T,S]/(\widetilde{P}(T,S))$ but $\varphi(C_+) = B_+$ so this is really a morphism $\operatorname{Spec}(B) \longrightarrow \operatorname{Proj}(A[T,S]/(\widetilde{P}(T,S))$ and by the local description in lemma 3.40, you can show that this is an isomorphism.