Aside from some trigonometric substitutions and identities, we will need the following identity, which can be shown using integration by parts twice:
$$
\int_0^{\infty}\cos(\alpha t)e^{-\lambda t}\,\mathrm{d}t=\frac{\lambda}{\alpha^2+\lambda^2}\tag{1}
$$
We will also use the standard arctangent integral:
$$
\int_0^\infty\frac{\mathrm{d}t}{a^2+t^2}=\frac\pi{2a}\tag{2}
$$
Now
$$
\begin{align}
&\left(\int_0^\infty\color{#C00000}{\sin}(x^2) e^{-\lambda x^2}\,\mathrm{d}x\right)^2\\
&=\int_0^\infty\int_0^\infty \color{#C00000}{\sin}(x^2)\color{#C00000}{\sin}(y^2) e^{-\lambda(x^2+y^2)}\,\mathrm{d}y\,\mathrm{d}x\tag{3.1}\\
&=\frac12\int_0^\infty\int_0^\infty \left(\cos(x^2-y^2) \color{#FF0000}{-}\cos(x^2+y^2)\right) e^{-\lambda(x^2+y^2)}\,\mathrm{d}y\,\mathrm{d}x\tag{3.2}\\
&=\frac12\int_0^{\pi/2}\int_0^\infty \left(\cos(r^2\cos(2\phi)) \color{#FF0000}{-}\cos(r^2)\right)e^{-\lambda r^2} \,r\,\mathrm{d}r\,\mathrm{d}\phi\tag{3.3}\\
&=\frac14\int_0^{\pi/2}\int_0^\infty \left(\cos(s\cos(2\phi)) \color{#FF0000}{-}\cos(s)\right) e^{-\lambda s} \,\mathrm{d}s\,\mathrm{d}\phi\tag{3.4}\\
&=\frac14\int_0^{\pi/2}\left(\frac{\lambda}{\cos^2(2\phi)+\lambda^2} \color{#FF0000}{-}\frac{\lambda}{1+\lambda^2}\right)\,\mathrm{d}\phi\tag{3.5}\\
&=\frac12\int_0^{\pi/4} \frac{\lambda}{\cos^2(2\phi)+\lambda^2}\,\mathrm{d}\phi \color{#FF0000}{-}\frac{\lambda\pi/8}{1+\lambda^2}\tag{3.6}\\
&=\frac14\int_0^{\pi/4} \frac{\lambda\,\mathrm{d}\tan(2\phi)} {1+\lambda^2+\lambda^2\tan^2(2\phi)} \color{#FF0000}{-}\frac{\lambda\pi/8}{1+\lambda^2}\tag{3.7}\\
&=\frac14\int_0^\infty\frac{\mathrm{d}t}{1+\lambda^2+t^2} \color{#FF0000}{-}\frac{\lambda\pi/8}{1+\lambda^2}\tag{3.8}\\
&=\frac{\pi/8}{\sqrt{1+\lambda^2}} \color{#FF0000}{-}\frac{\lambda\pi/8}{1+\lambda^2}\tag{3.9}
\end{align}
$$
$(3.1)$ change the square of the integral into a double integral
$(3.2)$ use $2\color{#C00000}{\sin}(A)\color{#C00000}{\sin}(B)=\cos(A-B)\color{#FF0000}{-}\cos(A+B)$
$(3.3)$ convert to polar coordinates
$(3.4)$ substitute $s=r^2$
$(3.5)$ apply $(1)$
$(3.6)$ pull out the constant and apply symmetry to reduce the domain of integration
$(3.7)$ multiply numerator and denominator by $\sec^2(2\phi)$
$(3.8)$ substitute $t=\lambda\tan(2\phi)$
$(3.9)$ apply $(2)$
Finally, take the square root of both sides of $(3)$ and let $\lambda\to0^+$ to get
$$
\int_0^\infty\color{#C00000}{\sin}(x^2)\,\mathrm{d}x=\sqrt{\frac\pi8}\tag{4}
$$
Addendum
I just noticed that the same proof works for
$$
\int_0^\infty\cos(x^2)\,\mathrm{d}x=\sqrt{\frac\pi8}\tag{5}
$$
if each red $\color{#C00000}{\sin}$ is changed to $\cos$ and each red $\color{#FF0000}{-}$ sign is changed to $+$.
We make use of the identity
$$ \sum_{n=-\infty}^{\infty} \frac{1}{a^{2} - (x + n\pi)^{2}} = \frac{\cot(x+a) - \cot(x-a)}{2a}, \quad a > 0 \text{ and } x \in \Bbb{R}. $$
Then for $\alpha, \beta > 0$ it follows that
\begin{align*}
I := \mathrm{PV}\int_{0}^{\infty} \frac{\log\cos^{2}(\alpha x)}{\beta^{2} - x^{2}}
&= \frac{1}{2} \mathrm{PV} \int_{-\infty}^{\infty} \frac{\log\cos^{2}(\alpha x)}{\beta^{2} - x^{2}} \, dx \\
&= \frac{\alpha}{2} \mathrm{PV} \int_{-\infty}^{\infty} \frac{\log\cos^{2}x}{(\alpha\beta)^{2} - x^{2}} \, dx \\
&= \frac{\alpha}{2} \sum_{n=-\infty}^{\infty} \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} \frac{\log\cos^{2}x}{(\alpha\beta)^{2} - (x+n\pi)^{2}} \, dx \\
&= \frac{\alpha}{2} \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} \left( \sum_{n=-\infty}^{\infty} \frac{1}{(\alpha\beta)^{2} - (x+n\pi)^{2}} \right) \log\cos^{2}x \, dx \\
&= \frac{1}{4\beta} \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} (\cot(x+\alpha\beta) - \cot(x-\alpha\beta)) \log\cos^{2}x \, dx,
\end{align*}
where interchanging the order of integration and summation is justified by Tonelli's theorem applied to the summation over large indices $n$. Then
\begin{align*}
I
&= \frac{1}{4\beta} \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} (\cot(x+\alpha\beta) - \cot(x-\alpha\beta)) \log\cos^{2}x \, dx \\
&= \frac{1}{2\beta} \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} (\cot(x+\alpha\beta) - \cot(x-\alpha\beta)) \log\left|2\cos x\right| \, dx \tag{1}
\end{align*}
Here, we exploited the following identity to derive (1).
$$ \mathrm{PV} \int_{-\frac{\pi}{2}}^{\frac{\pi}{2}} \cot(x+a) \, dx = 0 \quad \forall a \in \Bbb{R}. $$
Now with the substitution $z = e^{2ix}$ and $\omega = e^{2i\alpha\beta}$, it follows that
\begin{align*}
I
&= \frac{1}{2\beta} \Re \mathrm{PV} \int_{|z|=1} \left( \frac{\bar{\omega}}{z - \bar{\omega}} - \frac{\omega}{z - \omega} \right) \log(1 + z) \, \frac{dz}{z}. \tag{2}
\end{align*}
Now consider the following unit circular contour $C$ with two $\epsilon$-indents $\gamma_{\omega,\epsilon}$ and $\gamma_{\bar{\omega},\epsilon}$.
Then the integrand of (2)
$$ f(z) = \left( \frac{\bar{\omega}}{z - \bar{\omega}} - \frac{\omega}{z - \omega} \right) \frac{\log(1 + z)}{z} $$
is holomorphic inside $C$ (since the only possible singularity at $z = 0$ is removable) and has only logarithmic singularity at $z = -1$. So we have
$$ \oint_{C} f(z) \, dz = 0. $$
This shows that
\begin{align*}
I
&= \frac{1}{2\beta} \Re \lim_{\epsilon \downarrow 0} \left( \int_{-\gamma_{\omega,\epsilon}} f(z) \, dz + \int_{-\gamma_{\bar{\omega},\epsilon}} f(z) \, dz \right) \\
&= \frac{1}{2\beta} \Re \left( \pi i \mathrm{Res}_{z=\omega} f(z) + \pi i \mathrm{Res}_{z=\bar{\omega}} f(z) \right) \\
&= \frac{1}{2\beta} \Re \left( - \pi i \log(1 + \omega) + \pi i \log(1 + \bar{\omega}) \right) \\
&= \frac{\pi}{\beta} \arg(1 + \omega)
= \frac{\pi}{\beta} \arctan(\tan (\alpha \beta)).
\end{align*}
In particular, if $\alpha\beta < \frac{\pi}{2}$ then we have
$$ I = \pi \alpha. $$
But due to the periodicity of $\arg$ function, this function draws a scaled saw-tooth function for $\alpha > 0$. Of course, $I$ is an even function of both $\alpha$ and $\beta$, so the final result is obtained by even extension of this saw-tooth function.
Best Answer
You can also write it as: $$I=\int_0^1\frac{\log^2(x)}{x^2+1}dx+\int_1^\infty \frac{\log^2(x)}{x^2+1}dx$$ In the second one do a $x\rightarrow \frac{1}{x}$ $$\Rightarrow I=2\int_0^1 \frac{\ln^2 x}{1+x^2}dx =2\sum_{n=0}^\infty (-1)^n\int_0^1 x^{2n}\ln^2 xdx$$$$\int_0^1 t^a dt=\frac{1}{a+1}\Rightarrow \int_0^1 t^a \ln^k tdt=\frac{d^k}{da^k} \left(\frac{1}{a+1}\right)=\frac{(-1)^k k!}{(a+1)^{k+1}}$$ $$\Rightarrow I=4\sum_{n=0}^\infty \frac{(-1)^{n} }{(2n+1)^3}=4\cdot\frac{\pi^3}{32}=\frac{\pi^3}{8}$$