In this example from the book, it was assumed a particular solution was $y_p = A \cos 3x + B \sin 3x.$
The problem I want to solve is $\bf y'' + 4y = 3 \ \bf sin 2x$.
Solving the associated homogeneous equation $y'' + 4y = 0$, I get $y_c = c_1 \cos 2x + c_2 \sin 2x$.
Now for the $3 \sin 2x$ part, I followed the example and assumed $y_p = A \cos 2x + B \sin 2x$. I then worked out $y_p'' + 4y_p = 3 \sin 2x$ and got $y_p = -\frac{3}{4} \sin 2x$.
So my solution was $y = y_c + y_p = c_1 \cos 2x + c_2 \sin 2x -\frac{3}{4} \sin 2x$.
But the book has $y = c_1 \cos 2x + c_2 \sin 2x -\frac{3}{4} x \cos 2x$, because they assumed $y_p = A x \cos 2x + B x \sin 2x$.
How is it decided to use $y_p = A x \cos 2x + B x \sin 2x$ instead of $y_p = A \cos 2x + B \sin 2x$?
Best Answer
Since you have a homogeneous solution depenendent on $\sin (3x)$ and $\cos(3x)$ already, any linear combination of those plugged into the left-hand side of the ODE will produce $0$. (Because they are solutions of the homogeneous equation.)
So the trick is to scale the homogeneneous solution by a factor of $x$ and assume we are looking for solutions of the form $Ax \sin(3x) + Bx \cos(3x)$ as is in your book.