I am going to generalize the result in my post
$$\int xe^x\cos xdx =\frac{e^{x}}{2}[x \cos x+(x-1) \sin x]+C$$
by finding a reduction formula for
$$I_n=\int x^ne^x\cos xdx .$$
After trying for couple of hours, I found that it is hard to evaluate without its partner integral $$J_n=\int x^ne^x\sin xdx$$
Modified Version
As advised by Hans and Martin, I try to find the reduction formula (3) and (4) by complex numbers.
$$
\begin{aligned}
I_{n}+i J_{n} &=\int x^{n} e^{x}(\cos x+i \sin x) d x \\
&=\int x^{n} e^{x} \cdot e^{x i} d x \\
&=\int x^{n} e^{x(1+i)} d x \\
&=\frac{1}{1+i} \int x^{n} d\left(e^{(1+i) x}\right) \\
&=\frac{1}{1+i}\left[x^{n} e^{(1+i) x}-\int n x^{n-1} e^{(1+i) x} d x\right] \\
&=\frac{1}{1+i}\left[x^{n} e^{(1+i) x}-n\left(I_{n-1}+i J_{n-1}\right)\right] \\
&=\frac{1-i}{2}\left[x^{n} e^{x}(\cos x+i \sin x)-n\left(I_{n-1}+i J_{n-1}\right)\right]
\end{aligned}
$$
Now comparing the real and imaginary parts on both sides yields $$
\begin{array}{l}
I_{n}=\frac{1}{2}\left[x^{n} e^{x}(\cos x+\sin x)-n\left(I_{n-1}+J_{n-1}\right)\right] \\
J_{n}=\frac{1}{2}\left[x^{n} e^{x}(\sin x-\cos x)+n\left(I_{n-1}-J_{n-1}\right)\right]
\end{array}
$$
***Original Method ***
First of all, we need to evaluate the following integrals using integration by parts.
$$
I_0=\int e^{x} \cos x d x=\frac{e^{x}}{2}(\cos x+\sin x)+c_{1}
$$
and $$
J_0=\int e^{x} \sin x d x=\frac{e^{x}}{2}(\sin x-\cos x)+c_{2}
$$
Consequently, $$
\int e^{x}(\cos x +\sin x) d x=e^{x} \sin x+c_3
$$
and $$
\int e^{x}(\cos x-\sin x) d x=e^{x} \cos x+c_4
$$
We then obtain easily$$
\begin{aligned}
I_{n}+J_{n} &=\int x^{n} \cdot e^{x}(\cos x+\sin x) d x \\
&=\int x^{n} d\left(e^{x} \sin x\right) \\
&=x^{n} e^{x} \sin x-n \int x^{n-1} e^{x} \sin x d x \\
&=x^{n} e^{x} \sin x-n J_{n-1}\qquad \qquad\cdots (1)
\end{aligned}
$$
Similarly, $$
I_{n}-J_{n}=x^{n} e^{x} \cos x-n I_{n-1} \qquad\qquad \cdots(2)
$$
$(1)+(2) $ yields
$$
\boxed{I_{n}=\frac{1}{2}\left[x^{n} e^{x}(\cos x+\sin x)-n\left(I_{n-1}+J_{n-1}\right)\right]} \qquad \cdots (3)
$$
$(1)-(2) $ yields
$$\boxed{J_{n}=\frac{1}{2}\left[x^{n} e^{x}(\sin x-\cos x)+n\left(I_{n-1}-J_{n-1}\right)\right]}\qquad \cdots (4) $$
Now let’s try to find $I_2 $using $ (3) $ and $ (4).$
Using (3) yields
$$
\begin{aligned}
I_{1} &=\frac{1}{2}\left[x e^{x}(\cos x+\sin x)-\left(I_{0}+J_{0}\right)\right] \\
&=\frac{e^{x}}{2}[x \cos x+(x-1) \sin x]+C_1
\end{aligned}
$$
Using (4) yields
$$
J_{1}=\frac{e^{x}}{2}\left[x(\sin x-\cos x)+ \cos x\right]+C_2
$$
Using (3) again yields
$$
\begin{aligned}
I_{2} &=\frac{1}{2}\left[x^{2} e^{x}(\cos x+\sin x)-2\left(I_{1}+J_{1}\right)\right] \\
&=\frac{1}{2}\left[x^{2} e^{x} \cos x+x^{2} e^{x} \sin x-e^{x}(2 x \sin x-\sin x-\cos x)\right]\\&= \frac{e^{x}(x-1)}{2}[(x-1) \sin x+(x+1) \cos x]+C_3
\end{aligned}
$$
Using (4) again yields
$$
J_{2}=\frac{e^{x}(x-1)}{2}[(x+1) \sin x-(x-1) \cos x]+C_{4}
$$
My Question
Is there other alternative methods? You opinions and solutions are highly appreciated.
Best Answer
Note that
\begin{align} \int x^n e^{ax}dx =\frac{d^n}{da^n}\int e^{ax}dx =\frac{d^n}{da^n}\left(\frac{e^{ax}}a\right) = \sum_{k=0}^n \frac{(-1)^k n!}{(n-k)!} \frac{x^{n-k}e^{ax}}{a^{k+1}} \end{align} Then \begin{align} \int x^n e^{x}\cos x\>dx =&\>\Re \int x^n e^{(1+i)x}dx =\>\Re\sum_{k=0}^n \frac{(-1)^k n!}{(n-k)!} \frac{x^{n-k}e^{(1+i)x}}{(1+i)^{k+1}}\\ =&\sum_{k=0}^n \frac{(-1)^k n! x^{n-k}e^{x}}{2^{\frac{k+1}2}(n-k)!} \cos\bigg(x - \frac{(k+1)\pi}4 \bigg) \\ \\ \int x^n e^{x}\sin x\>dx =&\sum_{k=0}^n \frac{(-1)^k n! x^{n-k}e^{x}}{2^{\frac{k+1}2}(n-k)!} \sin\bigg(x - \frac{(k+1)\pi}4 \bigg) \end{align}