Let $\phi$ denote the Klein-Gordon field. Then its propagator $\langle 0 \mid [\phi(x), \phi(y)] \mid 0 \rangle$ can be calculated as
$$\int \frac{d^4}{(2\pi)^3} \frac{-e^{-ip(x-y)}}{p^2 -m ^2}. \tag{1}$$
Isolating just the $p^0$ part of this integral, which is where the problem is, we get
$$\int \frac{dp^0}{2\pi i} \frac{-e^{-p^0(x^0-y^0)}}{(p^0 + E_p)(p^0 -E_p)} \tag{2}$$
and so there are poles at $\pm E_m$.
The standard way of dealing with such an integral is of course to use Cauchy's residue theorem and choose a contour across the real line with small semi-circles of radius $\epsilon$ around the poles and then sending $\epsilon$ to zero in the limit. Thus no matter what contour we choose we should get the same answer.
What I am confused about is why we do not send $\epsilon \rightarrow 0$ when calculating these propagators. Since we do not do this we get different values for the integral (which depend on the choice of contour and how whether we push the poles above or below the real axis) and hence different propagators (advanced, retarded, and Feynman). I understand that the different propagators correspond to different supports (in terms of time), but unless we take $\epsilon \rightarrow 0$ we're no longer solving for $\langle 0 \mid [\phi(x), \phi(y)] \mid 0 \rangle$ but rather a modified version of it. What is going on here?
EDIT: To make things clearer, my question is the following: shouldn't the answer to (2) give rise to the same propagator in the limit $\epsilon \rightarrow 0$ regardless of what contour is used?
Best Answer
Here is a cute toy example from What is a Quantum Field Theory? A First Introduction for Mathematicians. M. Talagrand. CUP. Appendix N: Feynman Propagator and Klein-Gordon Equation, which hopefully clears some confusion.
Consider the (linear) differential operator $D(u):=\partial_t^2 u+m^2u$. We now seek fundamental solutions of $D$.
Depending on the boundary conditions we employ, we find different fundamental solutions. For example, the following functions, corresponding to the retarded, advanced and Feynman prescription: $$u_1(t)=\begin{cases} \displaystyle 0 \quad &\text{for} \quad t\leq 0 \\ \frac{1}{m} \sin(mt)\quad &\text{for} \quad t\geq0 \end{cases}$$
$$u_2(t)=\begin{cases} \displaystyle 0 \quad &\text{for} \quad t\geq 0 \\ -\frac{1}{m} \sin(mt)\quad &\text{for} \quad t\leq 0 \end{cases}$$
$$u_3(t)= i\frac{e^{-i|t|m}}{2m}\quad .$$
It should be easy to verify that these are indeed fundamental solutions of $D$.
To find these fundamental solutions, we might take the Fourier transform of $D(u)=\delta$ which gives $$-\omega^2 \hat u(\omega)+m^2 \hat u(\omega)=1\quad .$$
Naively inverting the above equation would give $$\hat u(\omega)=\frac{1}{-\omega^2+m^2} \quad , $$
which, with the inverse Fourier transform, would yield
$$u(t)=\int_\mathbb R\frac{\mathrm d \omega}{2\pi}\frac{e^{it\omega}}{-\omega^2+m^2} \quad . $$
The integral however does not make sense, the function is not defined at $\omega=\pm m$. But we can compute slightly different (well-defined) integrals.
For example by e.g. bypassing both poles from below, i.e. replacing $\omega \to \omega -i\epsilon$, with $\epsilon >0$: If $t<0$, the integral vanishes, and for $t>0$ it equals $\sin(mt)/m$, i.e. we obtain $u_1$. Similarly, bypassing the poles both from above gives $u_2$. Finally, you might bypass the poles by computing $$\int_\mathbb R\frac{\mathrm d \omega}{2\pi}\frac{e^{it\omega}}{-\omega^2+m^2-i\epsilon} \quad ,$$ which gives $u_3$.
Note that we've implicitly let $\epsilon \to 0^+$ in all cases after the integration.
Regarding your specific case, which is very similar to the toy example above: What can be shown is that the Feynman propagator, as a tempered distribution, is given by $$ \Delta_F(x)=\lim\limits_{\epsilon \to 0^+}\int \frac{\mathrm d^4p}{(2\pi)^3}\frac{\exp-i(x,p)}{-p^2+m^2-i\epsilon}\quad .$$
This is Lemma 13.10.4. (p. 370) in the same book.