A bit of background helps frame this question. The question itself is in the last sentence.
For his PhD thesis, Richard Feynman and his thesis adviser John Archibald Wheeler devised an astonishingly strange approach to explaining electron-electron interactions without using a field. Their solution was to take the everyday retarded wave solution to Maxwell's equations and mix it 50/50 with the advanced (backwards in time) solution that up until then had always been discarded as "obviously" violating temporal causality. In a series of papers they showed that this was not the case, and that the recoil of an electron when it emits a photon could self-consistently be explained as the result of an advanced photon traveling backwards in time and impacting the electron at the same instant in which the electron emits a forward-in-time photon.
While Feynman's thesis ideas deeply influenced his subsequent development of QED, e.g. in QED's backwards-in-time interpretation of antimatter electrons (positrons). Feynman in a letter to Wheeler later famously retracted (well, famously for some of us) the specific idea of paired photons traveling forwards and backwards in time. Feynman's specific reason for abandoning his thesis premise was vacuum polarization, which cannot be explained by direct electron-to-electron interactions. (Vacuum polarization is easily accommodated by QED, however.)
Feynman's abandonment of the original Feynman/Wheeler retarded/advanced photon scheme has always troubled me. The reason is this: If their original idea was completely invalid, the probability from an information correlation perspective of the idea leading to accurate predictions of how physics operates should have been vanishingly small. Instead, their oddly arbitrary 50/50 mix of real waves and hypothesized backwards-in-time waves almost works, all the way down to the minute length scales at which vacuum polarization becomes significant. One analogy is that the Feynman/Wheeler scheme behaves like a slightly broken mathematical symmetry, one that correctly describes reality over almost the entire range of phenomena to which it applies, but then breaks down at one of the extrema of its range.
My question, finally, is this: Does there exist a clear conceptual explanation, perhaps in the QED description of vacuum polarization for example, of why the Feynman/Wheeler retarded/advanced model of paired photons traveling two directions in time provides an accurate model of reality overall, despite its being incorrect when applied at very short distances?
Addendum 2012-05-30
If I've understood @RonMaimon correctly — and I certainly still do not fully understand the S-matrix part of his answer — his central answer to my question is both simple and highly satisfying: Feynman did not abandon the backward-forward scheme at all, but instead abandoned the experimentally incorrect idea that an electron cannot interact with itself. So, his objection to Wheeler could perhaps be paraphrased in a more upbeat form into something more like this: "Vacuum polarization shows that the electron does indeed interact with itself, so I was wrong about that. But your whole backwards-and-forwards in time idea works very well indeed — I got a Nobel Prize for it — so thank for pointing me in that direction!"
Answer to Ron, and my thanks.
Best Answer
The main important idea of Feynman Wheeler theory is to use propagators which are non-causal, that can go forward and backward in time. This makes no sense in the Hamiltonian framework, since the backward in time business requires a formalism that is not rigidly stepping from timestep to timestep. Once you give up on a Hamiltonian, you can also ask that the formalism be manifestly relativistically invariant. This led Feynman to the Lagrangian formalism, and the path integral.
The only reason the Feynman Wheeler idea doesn't work is simply because of the arbitrary idea that an electron doesn't act on itself, and this is silly. Why can't an electron emit and later absorb the same photon? Forbidding this is ridiculous, and creates a nonsense theory. This is why Feynman says he abandons the theory. But this was the motivating idea--- to get rid of the classical infinity by forbidding self-interaction. But the result was much deeper than the motivating idea.
Feynman never abandons the non-causal propagator, this is essential to the invariant particle picture that he creates later. But later, he makes a similar non-causal propagator for electrons, and figures out how to couple the quantum electrons to the photon without using local fields explicitly, beyond getting the classical limit right. This is a major tour-de-force, since he is essentially deriving QED from the requirement of relativistic invariance, unitarity, the spin of the photon and electron, plus gauge-invariance/minimal coupling (what we would call today the requirement of renormalizability). These arguments have been streamlined and extended since by Weiberg, you derive a quantum field theory from unitarity, relativistic invariance, plus a postulate on a small number of fundamental particles with a given spin<1.
In Feynman's full modern formalism, the propagators still go forward and backward in time just like the photon in Wheeler-Feynman, the antiparticle goes backward, and the particle forward (the photon is its own antiparticle). The original motivation for these discoveries is glossed over by Feynman a little, they come from Wheeler's focus on the S-matrix as the correct physical observable. Wheeler discovered the S-matrix in 1938, and always emphasized S-matrix centered computations. Feynman never was so gung-ho on S-matrix, and became an advocate of Schwinger style local fields, once he understood that the particle and field picture are complementary. He felt that the focus on S-matrix made him work much harder than he had to, he could have gotten the same results much easier (as Schwinger and Dyson did) using the extra physics of local fields.
So the only part of Wheeler-Feynman that Feynman abandoned is the idea that particles don't interact with themselves. Other than that, the Feynman formalism for QED is pretty much mathematically identical to the Wheeler-Feynman formalism for classical electrodynamics, except greatly expanded and correctly quantum. If Feynman hadn't started with backward in time propagation, it isn't clear the rest would have been so easy to formulate. The mathematical mucking around with non-causal propagators did produce the requisite breakthrough.
It must be noted that Schwinger also had the same non-causal propagators, which he explicitly parametrized by the particle proper time. He arrived at it by a different path, from local fields. However they were both scooped by Stueckelberg, who was the true father of the modern methods, and who was neglected for no good reason. Stueckelberg was also working with local fields. It was only Feynman, following Wheeler, who derived this essentially from a pure S-matrix picture, and the equivalence of the result to local fields made him and many others sure that S-matrix and local fields are simply two complementary ways to describe relativistic quantum physics.
This is not true, as string theory shows. There are pure S-matrix theories that are not equivalent to local quantum fields. Feynman was skeptical of strings, because they were S-matrix, and he didn't like S-matrix, having been burned by it in this way.