[Math] Euler-Lagrange, Gradient Descent, Heat Equation and Image Denoising

Question

For an image denoising problem, the author has a functional $E$ defined

$$E(u) = \iint_\Omega F \;\mathrm d\Omega$$

which he wants to minimize. $F$ is defined as

$$F = \|\nabla u \|^2 = u_x^2 + u_y^2$$

Then, the E-L equations are derived:

$$\frac{\partial E}{\partial u} = \frac{\partial F}{\partial u} –
\frac{\mathrm d}{\mathrm dx} \frac{\partial F}{\partial u_x} –
\frac{\mathrm d}{\mathrm dy} \frac{\partial F}{\partial u_y} = 0$$

Then it is mentioned that gradient descent method is used to minimize the functional $E$ by using

$$\frac{\partial u}{\partial t} = u_{xx} + u_{yy}$$

which is the heat equation. I understand both equations, and have solved the heat equation numerically before. I also worked with functionals. I do not understand however how the author jumps from the E-L equations to the gradient descent method. How is the time variable $t$ included? Any detailed derivation, proof on this relation would be welcome. I found some papers on the Net, the one by Colding et al. looked promising.

References:

http://arxiv.org/pdf/1102.1411 (Colding et al.)

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.117.1675&rep=rep1&type=pdf

http://dl.dropbox.com/u/1570604/tmp/functional-grad-descent.pdf

http://dl.dropbox.com/u/1570604/tmp/gelfand_var_time.ps (Gelfand and Romin)

Answer 1

You should note that a solution, $f$, to your differential equation, $\mathcal{L}[f] = 0$, is the steady state solution to the second equation, as $\partial_t f = 0$. By turning this into a parabolic equation, only the error term will depend on $t$, and it will decay with time. This can be seen by letting

$$h(x,y,t) = f(x,y) + \triangle f(x,y,t),$$

where $f$ is as before. Then

$$\mathcal{L}[h] = \mathcal{L}[\triangle f] = \partial_t \triangle f$$

In general, this method makes the equations amenable to minimization routines like steepest descent.

Edit: Since you mentioned that you wanted a book to reference, when I was taking numerical analysis, we used v. 3 of Numerical Mathematics and Computing by Cheney and Kincaid, and I found it very useful. Although, at points it lacked depth, however it provided a good jumping off point. They also have a more mathematically in depth book Numerical analysis: mathematics of scientific computing that may be useful to you, which I have not read.