Is the integral $$I = \int_0^\infty \text{d}x \, \exp\left( i x^2 \right)$$
well-defined? I would think it is, since I understand this object as the limit $$\lim_{a \rightarrow \infty} I(a)$$ where $$I(a) = \int_0^a \text{d}x \, \exp\left( i x^2 \right) = -\frac{\sqrt{\pi}}{2} e^{i \pi/4} \text{erf}\left( e^{3 i \pi/4} a \right)$$ which tends to the finite complex number $e^{i \pi/4} \sqrt{\pi}/2$ as $a \rightarrow \infty$. Yet I often read claims that $I$ is not well-defined, for example that
"it is an oscillatory integral which does not converge even schematically"
from this recent paper (top of p.4). (One of the authors has been awarded the Fields Medal, so I absolutely expect to be proven wrong here.) At this moment I don't see any essential difference with the integral
$$\tilde{I} = \int_0^\infty \text{d}x \, \exp\left( -x^2 \right) \,,$$ which I don't think anyone would claim does not converge or is ill-defined in any reasonable way. Of course one should define what is meant by this improper integral, but if we define it analogously as above we get the answer $\sqrt{\pi}/2$ that I hope we all agree upon.
Maybe I should add that by $\int_a^b \text{d}x \, f(x)$ for $a,b \in \mathbb{R}$ and with $f$ a (say) continuous complex-valued function $\mathbb{R} \rightarrow \mathbb{C}$, we mean $\int_a^b \text{d}x \, \text{Re}[f(x)] + i \int_a^b \text{d}x \, \text{Im}[f(x)]$. By the way, to me (at this point…) there's no mystery at all behind the integral $I$, just as there is none behind $\tilde{I}$. The real part is $\lim_{a \rightarrow \infty} \int_0^a \text{d}x \, \sin(x^2)$, and $\sin(x^2)$ oscillates faster and faster as $x$ grows while maintaining a constant amplitude, allowing for consecutive positive and negative contributions to $I(a)$ to cancel to a better and better degree as $a$ grows. If the oscillations are not fast enough this doesn't work. For instance, I would not claim that $$\int_0^\infty \text{d}x \, e^{ix}$$ is well-defined in the same way, since the limit $\lim_{a \rightarrow \infty} \int_0^a \text{d}x \, e^{ix}$ does not exist. (Although it is possible to make sense of $$\int_0^\infty \text{d}x \, e^{i \omega x} = \text{PV}(i/\omega) + \pi \delta(\omega)$$ using the notion of distribution.)
Best Answer
You have already established that $I$ is convergent in the sense that the limit of the integral over $[0,a]$ as $a \to \infty$ exists. This is apparent from the closed form valuation that you presented. Alternatively, making the variable change $u = x^2$, we have
$$\int_0^a e^{ix^2} \, dx = \int_0^{a^2} \frac{e^{iu}}{2\sqrt{u}}\, du,$$
and convergence as $a \to \infty$ follows from the Dirichlet test.
What sets $I$ and $\tilde{I}$ apart is that the former is conditionally convergent and the latter is absolutely convergent. The conditional convergence of $I$ is not particular troublesome as long as we make the strict interpretation as an improper Riemann integral
$$\int_0^\infty e^{ix^2} \, dx = \lim_{a \to \infty}\int_0^a e^{ix^2} \, dx, $$
and understand that the integrand here is not absolutely (or Lebesgue) integrable.
However, for integrands defined on $\mathbb{R}^n$ where $n > 1$, there is a more general way to define improper integrals (also called extended Riemann integrals) over arbitrary regions (both bounded and unbounded) where the integrand also may be unbounded. Here we define the extended integral over $A \subset \mathbb{R}^n$ as
$$\int_A f = \lim_{n \to \infty}\int_{C_n} f,$$
where $(C_n)$ is a sequence of compact rectifiable sets, increasing in the sense that $C_n \subset \text{int}(C_{n+1})$, and exhaustive in the sense that $\cup_{n=1}^\infty C_n = A$.
This is a natural way to define improper integrals over higher-dimensional regions that are not hyper-rectangles. It can be shown that if there exists one choice $(C_n)$ where we have,
$$\int_A |f| = \lim_{n \to \infty}\int_{C_n} |f| < +\infty,$$ then the integral $\int_A f$ is well-defined and independent of the particular choice for the sequence $(C_n).$
Returning to integration over subsets of $\mathbb{R}$, if the absolute convergence
$$\int_0^\infty |f| = \lim_{a \to \infty}\int_0^a|f| < +\infty$$ fails to hold, then it is not possible to define uniquely the improper integral
$$\int_0^\infty f(x) \, dx = \lim_{n \to \infty}\int_{C_n} f(x) \, dx,$$
for nested sequences $C_1 \subset C_2 \subset \ldots $ of compact sets such that $\cup_{n=1}^\infty C_n = [0,\infty)$.
An example with the integrand $f(x) = \frac{\sin x}{x}$ is given here.