There is no contradiction. The first exercise asks you to show that there exists (at least one) open set containing $A$ on which a smooth extension is defined. On the other hand, Lemma 2.26 is that for any open set $U\supseteq A$, a smooth extension exists. This is a much stronger statement, and hence requires a stronger assumption (that $A$ is closed).
So a counterexample for Exercise 2.27 will provide one open set $U\supseteq A$ on which no extension exists. But that does not rule out the possibility that there exists another open set $\tilde{U}\supseteq A$ for which Jeffrey Lee's Exercise hold.
Suppose $X \in \mathfrak{X}(M), Y \in \mathfrak{X}(N)$ we can define a vector field $X \oplus Y : M \times N \to T(M \times N)$ on product manifold $M \times N$ as
$$
(X \oplus Y)_{(p,q)} = (X_p,Y_q)
$$
under natural identification of $T_{(p,q)}(M \times N)$ with $T_p M \oplus T_qN$ (by isomorphism $\alpha : T_{(p,q)}(M \times N) \to T_pM \oplus T_qN$ defined as $\alpha (v) = (d\pi_M(v), d\pi_N(v))$, one can show that it is a smooth vector field on the product manifold.
So, wlog, given $X \in \mathfrak{X}(M_1)$ it can be checked that for any $X_j \in \mathfrak{X}(M_j)$ for $j=2,\dots,k$, the resulting product $X \oplus X_2 \oplus \cdots \oplus X_k$ is $\pi_1$-related to $X$ by the way the product vector field defined. So vector field on product manifold that $\pi_1$-related to $X$ is not unique. Of course we can choose $X \oplus \mathbf{0}\, \oplus \cdots\oplus \mathbf{0}$ for convenient.
Since you read Lee's, i want to point out that the construction of product vector field above is in fact an exercise in Lee's Introduction to Smooth Manifold (See Problem 8-17 and more general setting in Problem 8-18). However vector fields on the product manifold that $\pi_1$-related to a vector field $X \in \mathfrak{X}(M_1)$ is not necessarily in form of product vector field.
After read this post, i've come to conclusion that
$\mathfrak{X}(M \times N) \supsetneq \mathfrak{X}(M) \oplus \mathfrak{X}(N)$ (as shown in that answer),
Any vector vector field $V$ in product manifold $M \times N$ is in form of $V= X \oplus Y$ for some $X \in \mathfrak{X}(M)$ and $Y \in \mathfrak{X}(N)$ if and only if $V$ and $X$ are $\pi_M$-related and $V$ and $Y$ are $\pi_N$-related.
In more general setting, we know that for any smooth surjective submersion $F : M \to N$ and $X \in \mathfrak{X}(M)$, the pushforward $F_{*}(X)$ is a well-defined smooth vector field on $N$ that is $F$-related to $X$ is and only if $dF_p(X_p) = dF_q(X_q)$ whenever $p$ and $q$ are in the same fiber. So by applying this to the map $\pi_M : M \times N \to M$ and $\pi_N : M \times N \to N$, we have the following criteria :
Any vector vector field $V \in \mathfrak{X}(M \times N)$ is also in $\mathfrak{X}(M) \oplus \mathfrak{X}(N)$ if and only if $d\pi_M(V_{(p,q)})$ constant on each fiber $\{p\} \times N$ and $d\pi_N(V_{(p,q)})$ is constant on each fiber $M \times \{q\}$.
Best Answer
Allow me answer with notations that is more closely related to the book, which make it more pedantic than some answers here on MSE. Forgive me for that.
You have noticed that if we have a smooth vector field $Y \in \mathfrak{X}(M \times N)$ that is $F$-related to $X$, then for each point $p \in M$, $$ dF_p(X_p) = \alpha^{-1} \circ \alpha \circ dF_p(X_p) = \alpha^{-1}\big( X_p, df_p(X_p) \big) = Y_{(p,f(p))}, $$ with $\alpha : T_{(p,f(p))}(M \times N) \to T_pM \oplus T_{f(p)}N$ is the isomorphism $\alpha(v) = (d\pi_M(v),d\pi_N(v))$. So we need to find $Y \in \mathfrak{X}(M \times N)$ such that its values on the graph $$\Gamma_f = \{(p,q) \in M \times N \mid p \in M , q=f(p)\}$$ satisfy the relation above. Define a continuous vector field $Y : \Gamma_f \to T(M \times N)$ as $$ Y_{(p,f(p))} = dF_p(X_p). $$ Now we need to extend $Y$ to $M \times N$, by extending $Y$ locally around each point first. That is for each $(p,f(p)) \in \Gamma_f$ there is a neighbourhood $W$ of $(p,f(p))$ and a smooth vector field $\widetilde{Y}$ on $W$ such that $\widetilde{Y}|_{W \cap \Gamma_f} = Y|_{W \cap \Gamma_f}$. The hint is that write down the expression for $Y$ in a local coordinates first. Then decide what is the form of $\widetilde{Y}$. Here is the detail :