Motivation: The following problem arose in [1] while studying the vertical distribution of the zeros of the Riemann zeta-function. At the time, my collaborators and I were unable to solve it and I have never been able to derive a "satisfying answer."
Set-up: Let $r\ge 1$ and let $f \in L^2[0,1]$ be a continuous real-valued function of bounded variation on $[0,1]$, normalized so that
$$ \int_0^1(1-u)^{r^2-1}f(u)^2 du = 1. $$
Further define $M(c)=M(c,f,r)$ as
$$ M(c):=c+\frac{2 r}{\pi}\int_0^1 (1-u)^{r^2-1}f(u) \int_0^u \frac{\sin(\pi c v)}{v} f(u-v) \ dv \ du.$$
Question: How does one choose $r$ and $f$ optimally so that
$$ M(c) >1$$
for $c$ as small as possible?
An argument of Conrey, Ghosh, and Gonek [2] can be used to show that $M(c)<1$ if $c<\frac{1}{2}$ for any such $f$ and $r$. In [1], choosing $f$ to be a polynomial of low degree ($\le 6$) and using Mathematica to numerically optimize the $r$ and the coefficients, we were able to find $f$ and $r$ such that $M(.5155)>1$.
In the special case when $r=1$, Montgomery and Odlyzko [3] observed that this optimization problem had already been solved using prolate spheroidal wave functions (see comments below). Here is how they reduced the problem to one that was already solved. In this case, we have
$$ M(c) = c+ \int_0^1\int_0^1 f(u) f(v) \frac{\sin(\pi c(u-v))}{\pi(u-v)} \ dv \ du. $$
The double integral on the right-hand side is
$$ \int_{-c/2}^{c/2} \left| \int_0^1 f(v) e^{2\pi i t v} dv \right|^2 dt :=I(c),$$
say. They then observed that choosing
$$ f(x) = aR_{00}^{(1)}[\pi c/2,2x-1]$$
maximizes $I(c)$, where $R_{mn}^{(1)}[c,x]$ is the radial prolate spheroidal wave function of the first kind of order m and degree n, and a is a constant to be chosen according to our above normalization.
These wave functions are very hard to study numerically, and Montgomery and Odlyzko approximated them using modified Bessel functions. In [1], when $r=1$, we recovered their results to four decimal places using polynomials of degree four. So in this case it seems that polynomials of small degree work (almost) as well as more sophisticated techniques.
References:
[1] H. M. Bui, M. B. Milinovich, and N. C. Ng, A note on the gaps between consecutive zeros of the Riemann zeta-function, Proc. Amer. Math. Soc. 138 (2010), no. 12, pp. 4167-4175.
[2] J. B. Conrey, A. Ghosh, and S. M. Gonek, A note on gaps between zeros of the zeta function,
Bull. London Math. Soc. 16 (1984), 421–424.
[3] H. L. Montgomery and A. M. Odlyzko, Gaps between zeros of the zeta function, Colloq. Math.
Soc. Janos Bolyai, 34. Topics in Classical Number Theory (Budapest, 1981), North-Holland,
Amsterdam, 1984.
Edit/Additional Comments: The solution optimization problem when $r=1$ should probably be attributed to Slepian and Pollak and to Landau and Pollack in Prolate spheroidal wave functions, Fourier analysis and uncertainty I and II, Bell System Tech. J. (1961) 40, pp. 43-61 (I) and pp. 65-84 (II). Among other things, they prove the following results.
Theorem: Let $\alpha(c)$ be the least number such that
$$ \int_{-c/2}^{c/2} \left|\int_0^1 f(x) e^{2\pi i t x} dx \right|^2 dt \le \alpha(c) \int_0^1 |f(x)|^2 dx $$
for all $f\in L^2[0,1]$. Then $\alpha(c)$ is strictly increasing, $\alpha(c)\lt c$ for $c \gt 0$, $\alpha(c)\sim c$ as $c\to 0^+$, and $\alpha(c)\to 1$ as $c\to \infty$. Moreover, equality is achieved in the above inequality if $f(x) = R_{00}^{(1)}[\pi c/2,2x-1]$.
One of the possible hang-ups in solving the optimization problem when $r>1$ is that I have not been able to "complete" the double integral
$$\int_0^1 (1-u)^{r^2-1}f(u) \int_0^u \frac{\sin(\pi c v)}{\pi v} f(u-v) \ dv \ du$$
into a double integral of the form
$$ \int_0^1 \int_0^1 [\text{nice integrand}] \ dv \ du,$$
which is the first step in Montgomery & Odlyzko's argument.
Best Answer
You may set $v=t u$. When $v$ goes from 0 to $u$, $t$ goes from 0 to 1. So that your double integral becomes:
$$\int_0^1 (1-u)^{r^2-1}f(u) \int_0^1 \frac{\sin(\pi c t u)}{\pi t u} f(u(1-t))u \ dt \ du$$
Which is the format you are seeking for.