Let $S_{a, b, c, d, e} $ be the number of solutions with $x_1\ge a$, $x_2 \ge b$, $x_3\ge c$, $x_4\ge d$ and $x_5\ge e$.
The equation can be written as
$$(x_1-a)+(x_2-b)+(x_3-c)+(x_4-d)+(x_5-e)=21-a-b-c-d-e$$
So we have
$$S_{a, b, c, d, e} =\binom{21-a-b-c-d-e+4}{4}$$
if $a+b+c+d +e\le21$ and is $0$ otherwise.
The answer to this question is
$$S_{0, 1,15,0,0}-S_{4,1,15,0,0}-S_{0,4,15,0,0}+S_{4,4,15,0,0}=\binom{9}{4}-\binom{5}{4}-\binom{6}{4}+0=106$$
It might be more helpful to factorize the quadratic expression first (taking it as a variable in $x$). This way we can extract coefficient of $x^n$ ($u$ taken as a constant) and then coefficient of $u^m$ (in other words $[x^n u^m]f(x,u)=[u^m]([x^n]f(x,u))$. So, by factorization of denominator we arrive at
$$
\dfrac{1}{1-2x+x^2-ux^2}=\frac{1}{1-(1+\sqrt{u})x}\cdot \frac{1}{1-(1-\sqrt{u})x}
$$
which by geometric series is
$$
(\sum_{i \geq 0}(1+\sqrt{u})^ix^i) \cdot (\sum_{j \geq 0}(1-\sqrt{u})^j x^j ).
$$
So we get a coefficient of $x^n$
$$
\sum_{k=0}^{n}(1+\sqrt{u})^k(1-\sqrt{u})^{n-k}\tag{*}
$$
and the problem reduces to finding coefficient of $u^m$ in $(*)$. We can evaluate the expression for example by writing it as
$$
(1-\sqrt{u})^n\sum_{k=0}^{n}\left(\frac{1+\sqrt{u}}{1-\sqrt{u}}\right)^k
$$
and spot the finite geometric series with $q=\frac{1+\sqrt{u}}{1-\sqrt{u}}$, so we can just use well-known formula for the sum $\frac{q^{n+1}-1}{q-1}$. After some messy algebra we get
$$
\frac{1}{2\sqrt{u}}[(1+\sqrt{u})^{n+1}-(1-\sqrt{u})^{n+1}],
$$
which finally by Binomial theorem gives
$$
\frac{1}{2\sqrt{u}}\sum_{m=0}^{n+1}\binom{n+1}{m}\sqrt{u}^{m}(1-(-1)^{m}).
$$
For even $m$ the terms vanish and we are left with
$$
\sum_{m=0}^{\lfloor n/2 \rfloor}\binom{n+1}{2m+1}u^{m}.
$$
Now just read off the coefficient, also perhaps use $\binom{n}{k}=\binom{n}{n-k}$.
Best Answer
For $(b)$, note that the number of terms equals the number of solutions $(k_1,k_2,\ldots,k_{r-1})$ to $$k_1+k_2+\ldots+k_{r-1}\leq n$$ with each variable being a non-negative integer. Let $k_r=n-k_1-k_2-\ldots-k_{r-1}$. Then $(k_1,k_2,\ldots,k_r)$ is a non-negative integer solution to $$k_1+k_2+\ldots+k_r=n.$$ Hence the answers to $(a)$ and $(b)$ are the same.
An alternative way to see this is to observe that $$\left(1+y_1+y_2+\ldots+y_{r-1}\right)^n=\frac{(x_1+x_2+\ldots+x_r)^n}{x_r^n},$$ where $y_i=\frac{x_i}{x_r}$ for $i=1,2,\ldots,r-1$.