correlation-functionsgreens-functionsmathematical physicspath-integralpropagator
The kernel in path integral to transform the wave function (Eq 3.42 in Feynman and Hibbs, Quantum Mechanics and Path Integrals, emended edition):
$$\psi(x_b,t_b)=\int_{-\infty}^\infty K(x_b,t_b;x_c,t_c)\psi(x_c,t_c)dx_c$$
looks equivalent to the kernel to transform the integral:
$$(Tf)(u) = \int \limits_{t_1}^{t_2} K(t, u)\, f(t)\, dt$$
However, the table of transforms doesn't have any "Feynman transform". What exactly is it in mathematics?
I) Notational issues: Greens function vs. kernel. First of all, be aware that Ref. 1 between eq. (4-27) and eq. (4-28) effectively introduces the retarded Greens function/propagator
$$\begin{align} G(x_2,t_2;x_1,t_1)~=~&\theta(\Delta t)~K(x_2,t_2;x_1,t_1), \cr \Delta t~:=~&t_2-t_1,\end{align}\tag{A}$$
rather than the kernel/path integral
$$\begin{align} K(x_2,t_2;&x_1,t_1)\cr ~=~&\langle x_2,t_2 | x_1,t_1 \rangle\cr ~=~&\langle x_2|U(t_2,t_1)|x_1 \rangle\cr ~=~&\int_{x(t_1)=x_1}^{x(t_2)=x_2} \! {\cal D}x~ \exp\left[\frac{i}{\hbar}\int_{t_1}^{t_2} \!dt ~L\right] .\end{align}\tag{B} $$
Here $\theta$ denotes the Heaviside step function, and the Lagrangian
$$ L~:=~\frac{m}{2}\dot{x}^2-V(x)\tag{C}$$
is the Lagrangian for a non-relativistic point particle in 1 dimension with a potential $V$.
However, Ref. 1 confusingly denotes the Greens function $G$ with the same letter $K$ as the kernel! See also e.g. this and this Phys.SE posts. Therefore the eq. (4-29) in Ref. 1, which OP asks about, is better written as
$$\begin{align} D_2 G(x_2,t_2;x_1,t_1) ~=~&\delta(\Delta t)~\delta(\Delta x), \cr \Delta x~:=~&x_2-x_1, \end{align}\tag{D} $$
where we introduced the Schrödinger differential operator
$$\begin{align}D_2~:= ~&\frac{\partial}{\partial t_2} + \frac{i}{\hbar}\left(-\frac{\hbar^2}{2m}\frac{\partial^2}{\partial x_2^2}+V(x_2)\right)\cr ~=~&\frac{\partial}{\partial t_2} + \frac{\hbar}{i}\frac{1}{2m}\frac{\partial^2}{\partial x_2^2}+\frac{i}{\hbar}V(x_2).\end{align}\tag{E}$$
II) Proof of eq. (D). The sought-for eq. (D) follows directly from eq. (A) together with the following two properties (F) & (G) of the kernel $K$:
$$ D_2 K(x_2,t_2;x_1,t_1) ~=~0, \tag{F} $$
and
$$ K(x_2,t_2;x_1,t_1) ~\longrightarrow~\delta(\Delta x) \quad \text{for} \quad \Delta t \to 0^+. \tag{G}$$ $\Box$
III) So we can reformulate OP's question as follows.
Why the path integral (B) satisfies eqs. (F) & (G)?
Rather than going on a definition chase, perhaps the following heuristic derivation of eqs. (F) & (G) is the most convincing/satisfying/instructive. For sufficiently short times $|\Delta t| \ll \tau$, where $\tau$ is some characteristic time scale, i.e. in the diabatic limit, the particle only has time to feel an averaged effect of the potential $V$. So, using methods of Ref. 1, in that limit $|\Delta t| \ll \tau$, the path integral (B) reads
$$\begin{align} K(x_2,t_2;&x_1,t_1)\cr ~=~&\sqrt{\frac{m}{2\pi i\hbar \Delta t}} \exp\left\{ \frac{i}{\hbar}\left[ \frac{m}{2} \frac{(\Delta x)^2}{\Delta t}- \langle V\rangle \Delta t \right]\right\}, \end{align}\tag{H} $$
where the averaged potential is of the form
$$\begin{align} \langle V\rangle ~=~& V\left(\frac{x_1+x_2}{2}\right)+{\cal O}(\Delta x)\cr ~=~& V(x_2)+{\cal O}(\Delta x)\cr ~=~& V(x_1)+{\cal O}(\Delta x). \end{align}\tag{I}$$
IV) Proof of eq. (G). Note that it is implicitly assumed in eq. (H) that ${\rm Re}(i\Delta t)>0$ is slightly positive via the pertinent $i\epsilon$-prescription. Equation (G) then follows directly from eq. (H) via the heat kernel representation
$$ \delta(x)~=~ \lim_{|\alpha|\to \infty} \sqrt{\frac{\alpha}{\pi}} e^{-\alpha x^2},\qquad {\rm Re}(\alpha)~>~0, \tag{J} $$
of the Dirac delta distribution. $\Box$
V) Proof of eq. (F) for sufficiently small times $|\Delta t| \ll \tau$. It is a straightforward to check that eq. (H) satisfies the eq. (F) modulo contributions that vanish as $\Delta t\to 0$, cf. the following Lemma. $\Box$
Lemma. For sufficiently small times $|\Delta t| \ll \tau$, the path integral (H) satisfies $$ D_2 K(x_2,t_2;x_1,t_1) ~=~{\cal O}(\Delta t). \tag{K}$$
Sketched proof of eq. (K): Straightforward differentiation yields
$$ \begin{align} \frac{\partial}{\partial t_2} &K(x_2,t_2;x_1,t_1)\cr ~\stackrel{(H)}{=}~&-\left\{\frac{1}{2\Delta t} +\frac{i}{\hbar}\left[ \frac{m}{2} \left(\frac{\Delta x}{\Delta t}\right)^2+ \langle V\rangle +{\cal O}(\Delta t) \right]\right\} K(x_2,t_2;x_1,t_1),\end{align} \tag{L} $$
$$ \begin{align} \frac{\hbar}{i}\frac{\partial}{\partial x_2} &K(x_2,t_2;x_1,t_1)\cr ~\stackrel{(H)}{=}~&\left\{m \frac{\Delta x}{\Delta t} +{\cal O}(\Delta t) \right\} K(x_2,t_2;x_1,t_1), \end{align}\tag{M}$$
$$ \begin{align} \frac{\hbar}{i}\frac{1}{2m}&\frac{\partial^2}{\partial x_2^2} K(x_2,t_2;x_1,t_1) \cr ~\stackrel{(H)}{=}~&\left\{\frac{1}{2\Delta t} +\frac{i}{\hbar} \frac{m}{2} \left(\frac{\Delta x}{\Delta t}\right)^2 +{\cal O}(\Delta t) \right\} K(x_2,t_2;x_1,t_1). \end{align} \tag{N}$$
Also note that
$$ \begin{align} \left\{V(x_2)-\langle V\rangle \right\}K(x_2,t_2;x_1,t_1) ~\stackrel{(I)}{=}~&{\cal O}(\Delta x) K(x_2,t_2;x_1,t_1)\cr ~\stackrel{(G)}{=}~&{\cal O}(\Delta t),\end{align} \tag{O}$$
due to eqs. (I) and (G). The Lemma now follows by combining eqs. (E), (L), (N) & (O). $\Box$
VI) Proof of eq. (F) for large $\Delta t$. We use the path integral property
$$ \begin{align} K(x_2,t_2;&x_1,t_1)\cr ~=~&\int_{\mathbb{R}} \! dx_3~ K(x_2,t_2;x_3,t_3)~K(x_3,t_3;x_1,t_1),\end{align} \tag{2-31}$$
which is independent of the instant $t_3$. We now pinch the instant $t_3$ sufficiently close to the instant $t_2$, so that we can approximate the path integral $K(x_2,t_2;x_3,t_3)$ by the analog of eq. (H). If we apply the operator $D_2$ on the kernel, we get
$$ \begin{align}D_2 K(x_2,t_2;&x_1,t_1)\cr~\stackrel{(2-31)}{=}~&\int_{\mathbb{R}} \! dx_3~ D_2 K(x_2,t_2;x_3,t_3)~K(x_3,t_3;x_1,t_1) \cr ~\stackrel{(K)}{=}~&\int_{\mathbb{R}} \! dx_3~ {\cal O}(t_2-t_3)~K(x_3,t_3;x_1,t_1)\cr ~=~&{\cal O}(t_2-t_3) .\end{align}\tag{P}$$
Since the lhs. of eq. (P) does not depend on $t_3$, we conclude that it is zero. Hence eq. (F) also holds for large $\Delta t$ as well. $\Box$
References:
I) Conceptually, OP's original eq. (1)
$$\int_{\mathbb{R}}\! \mathrm{d}x_f~ \left| K(x_f,t_f;x_i,t_i) \right|^2 ~\stackrel{?}{=}~1 \qquad(\leftarrow\text{Wrong!})\tag{1} $$
clashes (as OP independently realized) with the fundamental principle of the Feynman path integral that the amplitude
$$K( x_f ,t_f ; x_i ,t_i )~=~\sum_{\rm hist.}\ldots$$
is a sum of histories, while the probability
$$P( x_f ,t_f ; x_i ,t_i )~=~|K( x_f ,t_f ; x_i ,t_i )|^2~\neq~\sum_{\rm hist.}\ldots $$
is not a sum of histories.
Concretely, the failure of eq. (1) may also be seen as follows. If we assume that$^1$
$$ K( x_i ,t_i ; x_f ,t_f ) ~=~ \overline{K( x_f ,t_f ; x_i ,t_i ) }, \tag{A}$$
and the (semi)group property of Feynman propagators/kernels
$$ K(x_f,t_f;x_i,t_i) ~=~ \int_{\mathbb{R}}\!\mathrm{d}x_m ~ K(x_f,t_f;x_m,t_m) K(x_m,t_m;x_i,t_i),\tag{B}$$
then the lhs. of OP's original first eq. (1) with $(x_i,t_i)=(x_f,t_f)$ is not equal to $1$, but instead becomes infinite
$$\begin{align} K(x_f,t_f;x_i,t_i) ~=~&\delta(x_f-x_i)~=~\delta(0)~=~\infty, \cr x_i~=~&x_f,\qquad t_i~=~t_f, \end{align}\tag{C}$$
because of OP's second formula (2).
II) The infinite normalization result (C) can be intuitively understood as follows. Recall that the paths in the path integral satisfy Dirichlet boundary condition $x(t_i)=x_i$ and $x(t_f)=x_f$. In other words, the particle is localized in $x$-position space at initial and final times. On the other hand, a particle localized in $x$-position space corresponds to a delta function wave function $\Psi(x)=\delta(x-x_0)$, which is not normalizable, cf. e.g this and this Phys.SE posts.
III) Conceptually, OP's first eq. (1')
$$\left| \int_{\mathbb{R}}\! \mathrm{d}x_f~K(x_f,t_f;x_i,t_i) \right| ~\stackrel{?}{=}~1 \quad(\leftarrow\text{Turns out to be ultimately wrong!}) \tag{1'} $$
is the statement that a particle that is initially localized at a spacetime event $(x_i,t_i)$ must with probability 100% be within $x$-space $\mathbb{R}$ at a final time $t_f$, as our QM model does not allow creation or annihilation of particles. However, such notion of absolute probabilities of the Feynman kernel $K(x_f,t_f;x_i,t_i)$ cannot be maintained when a concept has to be converted into mathematical formulas, as discussed in detail in this Phys.SE post. In general, OP's first eq. (1') only holds for short times $\Delta t \ll \tau$, where $\tau$ is some characteristic time scale of the system.
IV) Example. Finally, let us consider the example of a non-relativistic free particle in 1D. The Feynman propagator then reads
$$\begin{align} K( x_f ,t_f ; x_i ,t_i ) ~=~& \sqrt{\frac{A}{\pi}} e^{-A(\Delta x)^2}\cr ~=~& \sqrt{\frac{m}{2\pi i\hbar} \frac{1}{\Delta t}} \exp\left[ \frac{im}{2\hbar}\frac{(\Delta x)^2}{\Delta t}\right],\cr A~:=~&\frac{m}{2 i\hbar} \frac{1}{\Delta t} , \cr \Delta x~:=~&x_f-x_i, \cr \Delta t~:=~&t_f-t_i ~\neq ~0. \end{align}\tag{D}$$
[It is an instructive exercise to show that formula (D) satisfies eqs. (A-C) and OP's second formula (2).] The Gaussian integral over $x_f$ is one
$$ \int_{\mathbb{R}}\!\mathrm{d}x_f ~ K(x_f,t_f;x_i,t_i)~=~1,\tag{E} $$
which shows that OP's first eq. (1') actually holds for a free particle. The integrand
$$ \begin{align} |K(x_f,t_f;x_i,t_i)|^2~=~& \frac{|A|}{\pi}~=~ \frac{m}{2\pi \hbar}\frac{1}{|\Delta t|}, \cr \Delta t ~\neq ~&0,\end{align}\tag{F}$$
on the lhs. of OP's original first eq. (1) is independent of the midpoint $x_m$. Hence the integral over $x_m$ (i.e. lhs. of OP's first eq. (1)) becomes infinite
$$\begin{align} \int_{\mathbb{R}}\!\mathrm{d}x_f ~ |K(x_f,t_f;x_i,t_i)|^2~=~& \frac{m}{2\pi \hbar}\frac{1}{|\Delta t|} \int_{\mathbb{R}}\!\mathrm{d}x_f ~=~\infty, \cr \Delta t ~\neq ~&0,\end{align}\tag{G} $$
in agreement with what we found in eq. (C) in section I.
References:
--
$^1$ Note that Ref. 1 defines $K(x_f,t_f;x_i,t_i)=0$ if $t_i>t_f$, see Ref. 1 between eq. (4-27) and eq. (4-28). Here we assume property (A) instead.
Best Answer
For a given quantum system, the kernel of the path integral is, in fact, the kernel of an integral transform as you explicitly write down. It is the transform that governs time evolution of the system as is manifest in your first equation. For this reason, it is often referred to as the propagator of a given system.
For example, for a single, non-relativistic particle moving on some portion of the real line with a constant Hamiltonian (and thus unitary evolution), the kernal and unitary time-evolution operator are related as follows: \begin{align} K(x,t; x', t') = \langle x|U(t,t')|x'\rangle. \end{align} Kernels for distinct quantum systems will, in general, be different, because these systems have different hamiltonians and thus different behaviors under time evolution. Contrast this to the e.g. the Fourier and Laplace transforms whose kernels are always the same (up to pesky conventions of course).
For example, the kernel for a free massive, non-relativistic particle moving on the real line is \begin{align} K(x_b t_b; x_a, t_a) = \left[\frac{2\pi i\hbar(t_b-t_a)}{m}\right]^{-1/2}\exp\frac{im(x_b-x_a)^2}{2\hbar(t_b-t_a)} \end{align} and you could look up the kernels of a number of other systems (like the harmonic oscillator).
The main point. There is a different "Feynman transform," as you put it, for each quantum system. This is why you won't find just one in a table of mathematical transforms under some such name. This does bring up another interesting question though: is there a table of known kernels for various quantum systems somewhere? I'd be interested to know myself!
More on specific kernels and some generalizations here: http://en.wikipedia.org/wiki/Propagator