In this book, the functional derivative of a functional $F[f]$ is defined as
$$\frac{\delta F}{\delta f(x)} = \lim_{\epsilon\rightarrow0} \frac{F[f(y)+\epsilon \delta(y-x)] – F[f(y)]}{\epsilon}$$
It then proceeds to say that the functional derivative tells you how the number returned by the functional $F[f(x)]$ changes as you slightly change the function $f(x)$ that you feed into the machine (the author defined a functional as a machine that returns a number if given an input function).
Based on this explanation of the functional derivative, shouldn't its definition be something like
$$\frac{\delta F}{\delta f(x)} = \lim_{\epsilon\rightarrow0} \frac{F[f(y)+\epsilon \delta(y)] – F[f(y)]}{\epsilon}$$
instead? Now it makes sense because the function feeded into the machine is slightly changed to $f^{'}(x)=f(x)+\epsilon \delta(x)$.
Why is there an extra variable in the $\epsilon \delta(y-x)$ term in the author's definition of the functional derivative?
Best Answer
I don't know if this qualifies as an answer but let me give it a shot. For functionals such as
\begin{align} F\left[y(x)\right]=\int_{0}^{1}y^{2}(x)\mathrm{d}x \end{align}
We are familiar with determining the first variation
\begin{align} \delta F=F\left[y+\delta y\right]-F\left[y\right]+\mathcal{O}\left(||\delta y||^{2}\right) \end{align}
For this problem $\delta F=\int_{0}^{1} 2y\delta y\mathrm{d}x$. We want to define the functional derivative $\delta F/\delta y$ in the following way \begin{align} \delta F =\int_{0}^{1}\frac{\delta F}{\delta y}\delta y\mathrm{d}x \end{align}
To finish this explanation I now make a change of notation $\delta y=\varepsilon \psi(x)$, I will also change the variable of integration $x\to s$.
\begin{align} F\left[y+\varepsilon\psi\right]-F\left[y\right]=\varepsilon\int_{0}^{1}\frac{\delta F}{\delta y}(s)\psi(s)\mathrm{d}s+\mathcal{O}(\varepsilon^{2}) \end{align}
\begin{align*} \int_{0}^{1}\frac{\delta F}{\delta y}(s)\psi(s)\mathrm{d}s=\lim_{\varepsilon\to 0}\frac{F\left[y+\varepsilon\psi\right]-F\left[y\right]}{\varepsilon} \end{align*}
To obtain $(\delta F/\delta y)(x)$, we simply need to use the sampling function $\psi(s)=\delta(s-x)$.
\begin{align*} \frac{\delta F}{\delta y}(x)=\int_{0}^{1}\frac{\delta F}{\delta y}(s)\delta(s-x)\mathrm{d}s=\lim_{\varepsilon\to 0}\frac{F\left[y+\varepsilon\delta(s-x)\right]-F\left[y\right]}{\varepsilon} \end{align*}
I hope that helps.