The $L^\infty$ statement is definitely not true. For instance, take $\Omega = (0,1]$, and let $$\mathcal{F}_n = \sigma(((k-1)2^{-n}, k2^{-n}] : 1 \le k \le 2^n)$$ be the filtration generated by intervals whose endpoints are multiples of $2^{-n}$, so $\mathcal{F}$ is the Borel $\sigma$-algebra. Let $P$ be Lebesgue measure. Note every $\mathcal{F}_n$-measurable random variable is a step function. Now let $X = 1_A$ where $A$ is a Borel set that has positive but not full measure in every interval; see Construction of a Borel set with positive but not full measure in each interval. Then $X$ is $\mathcal{F}$-measurable, but $\|X - E[X \mid \mathcal{F}_n]\|_{L^\infty} \ge 1/2$ for every $n$.
Alternatively, note that the space $L^\infty(\mathcal{F}_n)$ of bounded $\mathcal{F}_n$-measurable functions is finite dimensional and thus separable in the $L^\infty$ norm. If your claim were true then $\bigcup_n L^\infty(\mathcal{F}_n)$ would be dense in $L^\infty$ and thus $L^\infty$ would be separable, which it isn't.
Or, if you want a more probabilistic example: suppose we have a sequence of independent biased coin flips, where $A_n$ is the event that the $n$th coin is heads. Suppose that $P(A_n) > 0$ for all $n$ and that $\sum_n P(A_n) < \infty$. Let $Y = \sum_n 1_{A_n}$ be the total number of heads observed; by Borel-Cantelli $Y < \infty$ almost surely. Let $X = 1$ on the event that $Y$ is odd and $0$ otherwise. If $\mathcal{F}_n = \sigma(A_1, \dots, A_n)$ is the natural filtration, then it is clear that on each nonempty event in $\mathcal{F}_n$, the random variable $X$ takes both values 0 and 1 with positive probability (no matter how the first $n$ flips came out, the remaining number of heads could be either even or odd and thus both parities are still possible for $Y$). So the $L^\infty$ distance between $X$ and any $\mathcal{F}_n$-measurable random variable must be at least $1/2$.
So the claim from the book (could you give the complete citation?) seems to be an error.
As for $p<1$, there is a basic problem that conditional expectation $E[X \mid \mathcal{F}_n]$ only makes sense when $X$ is integrable, so in general it has no meaning if we only know $X \in L^p$, $p < 1$. You could ask for $X$ to be nonnegative, but in that case we would typically get $P(E[X \mid \mathcal{F}_n] = \infty) > 0$ and so $E[X \mid \mathcal{F}_n]$ would generally not be in $L^p$ at all. For instance, take the same filtered probability space on $(0,1]$ as above, take $p=1/2$ for concreteness, and take $X(x) = 1/x$ which is in $L^{1/2}$. Then $E[X \mid \mathcal{F}_n]$ is equal to $\int_0^{2^{-n}} \frac{1}{x}\,dx = \infty$ a.e. on the set $(0, 2^{-n}]$, and thus $E[E[X \mid \mathcal{F}_n]^{1/2}] = \infty$, so it is not in $L^{1/2}$. There is thus no hope of having $E[X \mid \mathcal{F}_n] \to X$ in $L^{1/2}$.
I think almost sure convergence doesn't work either, even assuming $X$ is nonnegative. Again work on the same filtered probability space on $(0,1]$. Fix a countable dense subset $x_k$ of $(0,1]$ and let $X(x) = \sum_{k=1}^\infty 2^{-k} \frac{1}{|x-x_k|}$. Since every interval contains a spike like $1/|x|$, we have $E[X \mid \mathcal{F}_n] = +\infty$ almost surely for every $n$. However, the Pythagorean theorem and triangle inequality gives $\sqrt{\sum a_k} \le \sum \sqrt{a_k}$, so we have $$X(x)^{1/2} \le \sum_{k=1}^\infty 2^{k/2} \frac{1}{\sqrt{|x-x_k|}}$$
from which I think it follows that $E[X^{1/2}] < \infty$ and in particular $X < \infty$ almost surely.
Best Answer
We have to show that if $ X\in\mathbb L^p$ and $X_n:=\mathbb E[X\mid \mathcal F_n]$, then $\mathbb E|X_n-\mathbb E[X\mid \mathcal F_\infty|^p\to 0$.
We can use a truncation argument: for a fixed $R$ we have $$\mathbb E|X_n-Y|^p\leqslant 2^{p-1}\mathbb E\left|\mathbb E[X\chi\{|X|\leqslant R\}\mid\mathcal F_n]-\mathbb E[X\chi\{|X|\leqslant R\}\mid\mathcal F_\infty]\right|^p+\\ +2^{p-1}\mathbb E\left|\mathbb E[X\chi\{|X|\gt R\}\mid\mathcal F_n]-\mathbb E[X\chi\{|X|\gt R\}\mid\mathcal F_\infty]\right|^p.$$ Since $|\mathbb E[X\chi\{|X|\leqslant R\}\mid\mathcal F_n]-\mathbb E[X\chi\{|X|\leqslant R\}\mid\mathcal F_\infty]|$ is bounded by $2R$, we get $$\mathbb E|X_n-Y|^p\leqslant 2^{p-1}(2R)^{p-1}\mathbb E\left|\mathbb E[X\chi\{|X|\leqslant R\}\mid\mathcal F_n]-\mathbb E[X\chi\{|X|\leqslant R\}\mid\mathcal F_\infty]\right|\\ +4^{p-1}\mathbb E|X\chi\{|X|\gt R\}|^p.$$ Using the $\mathbb L^1$ case with $Y:=X \chi\{|X|\leqslant R\}$, we obtain $$\limsup_{n\to \infty}\mathbb E|X_n-Y|^p\leqslant 4^{p-1}\mathbb E|X\chi\{|X|\gt R\}|^p.$$ Since this bound is true for any $R$, we obtain the wanted conclusion by monotone convergence.