I am assuming your difficulty is in the jump between Eq.2 and Eq.3. All you need is an Euler-Lagrange equation, as in their equation (3). In their notation $f(x,y,\dot y)$ would be your $\int \{y(x)-t\}^2p(x,t)dx$, so that $df/d\dot{y}=0$, for instance.
Let $I$ be the indicator function: it is equal to $1$ for true arguments and $0$ otherwise. Pick $0\lt\alpha\lt 1$ and set
$$\Lambda_\alpha(x)=\alpha x\, I(x\ge 0) - (1-\alpha)x\, I(x\lt 0).$$
This figure plots $\Lambda_{1/5}$. It uses an accurate aspect ratio to help you gauge the slopes, which equal $-4/5$ on the left side and $+1/5$ on the right. In this case excursions above $0$ are heavily downweighted compared to excursions below $0$.
This is a natural function to try because it weights values $x$ that exceed $0$ differently than $x$ that are less than $0$. Let's compute the associated loss and then optimize it.
Writing $F$ for the distribution function of $X$ and setting $L_\alpha(m,x) = \Lambda_\alpha(x-m)$, compute
$$\eqalign{
\mathbb{E}_F(L_\alpha(m,X))&=\int_\mathbb{R} \Lambda_\alpha(x-m)dF(x)\\
&=\alpha\int_\mathbb{R} I(x\ge m)(x-m) dF(x) - (1-\alpha)\int_\mathbb{R} (x-m)I(x\lt m) dF(x)\\
&=\alpha\int_m^\infty(x-m)dF(x) - (1-\alpha)\int_{-\infty}^m(x-m) dF(x).
}$$
As $m$ varies in this illustration with the Standard Normal distribution $F$, the total probability-weighted area of $\Lambda_{1/5}$ is plotted. (The curve is the graph of $\Lambda_{1/5}(x-m)dF(x)$.) The right-hand plot for $m=0$ most clearly shows the effect of downweighting the positive values, for without this downweighting the plot would be symmetric about the origin. The middle plot shows the optimum, where the total amount of blue ink (representing $\mathbb{E}_F(L_{1/5}(m,X))\ $) is as small as possible.
This function is differentiable and so its extrema can be found by inspecting the critical points. Applying the Chain Rule and the Fundamental Theorem of Calculus to obtain the derivative with respect to $m$ gives
$$\eqalign{
\frac{\partial}{\partial m}\mathbb{E}_F(L_\alpha(m,X))&=\alpha\left(0-\int_m^\infty dF(x)\right) - (1-\alpha)\left(0 - \int_{-\infty}^m dF(x)\right)\\
&= F(m) - \alpha.
}$$
For continuous distributions this always has a solution $m$ which, by definition, is any $\alpha$ quantile of $X$. For non-continuous distributions this might not have a solution but there will be at least one $m$ for which $F(x)-\alpha\lt 0$ for all $x\lt m$ and $F(x)-\alpha\ge 0$ for all $x\ge m$: this also (by definition) is an $\alpha$ quantile of $X$.
Finally, because $\alpha\ne 0$ and $\alpha\ne 1$, it is clear that neither $m\to-\infty$ nor $m\to\infty$ will minimize this loss. That exhausts the inspection of the critical points, showing that $\Lambda_\alpha$ fits the bill.
As a special case, $\mathbb{E}_F(2L_{1/2}(m,X)) = \mathbb{E}_F\left(\left|m-x\right|\right)$ is the loss exhibited in the question.
Best Answer
I would like to explain the way I understood it, explaining each and every step on the way.
Assumptions:
Basic formulas:
$$\mathbb{E}_t[g|x] = \int_t{g(x,t)p(t|x)\mathop{dt}} \ (\mathbb{E}_t[g|x] \text{ is a function of $x$ and constant w.r.t. } t) \tag{1}\label{1} $$ $$\mathbb{E}_t[t|x] = \int_t{t.p(t|x)\mathop{dt}} \tag{2}\label{2}$$ $\operatorname{var}_t[t|x] = \int_t{(t - \mathbb{E}_t[t|x])^2p(t|x)\mathop{dt}} = \mathbb{E}_t[(t - \mathbb{E}_t[t|x])^2 | x] \tag{3}\label{3}$ $$ \eqalign{\mathbb{E}_t[f(x)g(x,t)|x] &= \int_t{f(x)g(x,t)p(t|x)\mathop{dt}} \\ &= f(x)\int_t{g(x,t)p(t|x)\mathop{dt}} \\ &= f(x) \ \mathbb{E}_t[g|x] } \tag{4}\label{4}$$ $$\mathbb{E}_t[f(x)|x] = f(x) \tag{4a}\label{4a}$$ $$\mathbb{E}_{x,t}[g] = \mathbb{E}_x[\mathbb{E}_t[g|x]] \tag{5}\label{5}$$
We derive the last formula above. $$ \eqalign{ \mathbb{E}_{x,t}[g] &= \int_x\int_tg(x,t)p(x,t)\mathop{dx}\mathop{dt}\\ &= \int_x\int_tg(x,t)p(x)p(t|x)\mathop{dx}\mathop{dt}\\ &= \int_x p(x)\int_t g(x,t)p(t|x)\mathop{dt}\mathop{dx}\\ &= \int_x \mathbb{E}_t[g|x]p(x)\mathop{dx} \text{ (using \ref{1}) } \\ &= \mathbb{E}_x [\mathbb{E}_t[g|x]] \\ } $$
Derivation of the expected loss:
Represent the Loss function in the form as below. Please notice the subscript $t$ in the $\mathbb{E}_t$ notations. This was omitted in the book, but I added it here for clarity. $$ \eqalign{ L(x,t) &= (y(x)-t)^{2} \\ &= (y(x) - \mathbb{E}_t[t|x])^{2} + 2(y(x) - \mathbb{E}_t[t|x])(\mathbb{E}_t[t|x]-t) + (\mathbb{E}_t[t|x]-t)^{2} \\ &= L_1 + 2L_2 + L_3 } $$
Hence the joint expectation can be represented as:
$$ \eqalign{ \mathbb{E}_{x,t}[L] &= \mathbb{E}_{x,t}[L_1] + 2\mathbb{E}_{x,t}[L_2] + \mathbb{E}_{x,t}[L_3] } $$
We derive the 3 expectations: $$ \eqalign{ \mathbb{E}_{x,t}[L_1] &= \mathbb{E}_{x,t}[(y(x) - \mathbb{E}_t[t|x])^{2}] \\ &= \mathbb{E}_x[ \ \mathbb{E}_t[(y(x) - \mathbb{E}_t[t|x])^{2} | x] \ ] \ \ \text{ (using \ref{5}) } \\ &= \mathbb{E}_x[(y(x) - \mathbb{E}_t[t|x] )^{2}] \text{ (using \ref{4a}, as the operand is a function of $x$ only)} \\ } $$
$$ \eqalign{ \mathbb{E}_{x,t}[L_2] &= \mathbb{E}_{x,t}[(y(x) - \mathbb{E}_t[t|x])(\mathbb{E}_t[t|x]-t)] \\ &= \mathbb{E}_x[ \ \mathbb{E}_t[\{(y(x) - \mathbb{E}_t[t|x])(\mathbb{E}_t[t|x]-t)\} \ | \ x] \ ] \text{ (using \ref{5}) } \\ &= \mathbb{E}_x[ \ (y(x) - \mathbb{E}_t[t|x]) \ \mathbb{E}_t[(\mathbb{E}_t[t|x]-t) | x] \ \ ] \ \text{ (using \ref{4} on the inner expectation)} } $$
Considering only the inner expectation: $$ \eqalign{ \mathbb{E}_t[(\mathbb{E}_t[t|x]-t) | x] &= \mathbb{E}_t[\mathbb{E}_t[t|x] | x] - \mathbb{E}_t[t|x] \text{ (using inearity of $\mathbb{E}$)} \\ &= \mathbb{E}_t[t|x] - \mathbb{E}_t[t|x] \text{ (using \ref{4a} as $\mathbb{E}_t[t|x]$ is a function of $x$)}\\ &= 0 } $$
Therefore, $$ \eqalign{ \mathbb{E}_{x,t}[L_2] &= \mathbb{E}_x[ \ (y(x) - \mathbb{E}_t[t|x]) \ \cdot \ 0 \ ] \\ &= 0 } $$
$$ \eqalign{ \mathbb{E}_{x,t}[L_3] &= \mathbb{E}_{x,t}[(\mathbb{E}_t[t|x]-t)^{2}] \\ &= \mathbb{E}_x[ \ \mathbb{E}_t[(\mathbb{E}_t[t|x]-t)^{2} | x] \ ] \text{ (using \ref{5}) } \\ &= \mathbb{E}_x[\operatorname{var}_t[t|x]] \text{ (using \ref{3}) } } $$
Putting them all together and expressing the $\mathbb{E}_x$ terms as integrals under $x$, we get the following form:
$$ \mathbb{E}_{x,t}[L] = \int_x (y(x) - \mathbb{E}_t[t|x])^2 p(x)\mathop{dx} + \int_x \operatorname{var}_t[t|x] p(x) \mathop{dx} $$
Note: As mentioned by @Juho Kokkalla, the erroneous last term in the book is corrected in the errata.