$\newcommand{\la}{\lambda}\newcommand{\Ga}{\Gamma}$Write
\begin{equation*}
I(\la)=\int_0^\infty dx\,f(x)J(\la x),
\end{equation*}
where
\begin{equation*}
J(z):=\int_0^1 dt\,e^{i z t^a}=\frac b{z^b}K(z),
\end{equation*}
\begin{equation*}
K(z):=\int_0^z du\,u^{b-1}e^{i u},
\end{equation*}
$b:=1/a\in(0,1)$.
Note that
\begin{equation*}
K(z)\to\int_0^\infty du\,u^{b-1}e^{i u}=(-i)^{-b}\Ga(b) \tag{0}\label{0}
\end{equation*}
as $z\to\infty$. (The equality in \eqref{0} can be obtained in a number of ways; in particular, it follows immediately from formulas 3.761.4 and 3.761.9 of Gradshteyn and Ryzhik, 7th Edition.)
Also, since $u^{b-1}$ decreases to $0$ as $u$ increases from $0$ to $\infty$, we have $|K(z)|\le C$ for some real $C>0$ and all real $z\ge0$.
So, letting $\la\to\infty$, by dominated convergence we get
\begin{equation*}
\la^b I(\la)=\int_0^\infty dx\,f(x)\frac b{x^b}K(\la x) \\
\to (-i)^{-b}\Ga(b+1)\int_0^\infty dx\,f(x)/x^b,
\end{equation*}
so that
\begin{equation*}
I(\la)\sim R(\la):=(-i\la)^{-b}\,\Ga(b+1)\int_0^\infty dx\,f(x)/x^b. \tag{1}\label{1}
\end{equation*}
Here are the graphs $\{(\la,\Re\frac{I(\la)}{R(\la)})\colon0<\la\le20\}$ (black) and $\{(\la,\Im\frac{I(\la)}{R(\la)})\colon0<\la\le20\}$ (blue) for $a=2.1$ and $f(x)=\exp(-\frac1{x(1-x)})\,1(0<x<1)$, which confirm \eqref{1}:
$\newcommand{\R}{\mathbb R}\newcommand\sgn{\operatorname{sgn}}\newcommand{\vpi}{\varphi}$Obviously, for $a:=\sqrt{2\pi}\,\psi(0)$ we have
\begin{equation*}
\psi(0)=f(0),
\end{equation*}
where
\begin{equation*}
f:=a\vpi
\end{equation*}
and $\vpi$ is the standard normal density. Letting now $g(x):=\frac{\psi(x)-f(x)}x$ for $x\ne0$, with $g(0):=\psi'(0)$, we see that $g$ is a smooth integrable function and
\begin{equation*}
\psi(x)=f(x)+xg(x) \tag{1}\label{1}
\end{equation*}
for all real $x$.
So,
\begin{equation*}
I(t)=\frac{J_f(t)+J_h(t)}{\sqrt t}, \tag{2}\label{2}
\end{equation*}
where $h(x):=xg(x)$,
\begin{equation*}
J_f(t):=
\int_0^{\sqrt t}\frac{dy}{(\frac y{\sqrt t}-z_1)(\frac y{\sqrt t}-z_2)}
\int_{\R}dx\,f(x)e^{ix^2y^2},
\end{equation*}
\begin{equation*}
z_1:=e^{i\theta_1},\quad z_2:=e^{i\theta_2},
\end{equation*}
with $J_h(t)$ defined similarly. Here and in what follows, $t$ is any real number $>0$, unless specified otherwise.
Note that
\begin{equation*}
\Im z_1\ne0\quad\text{and}\quad\Im z_2\ne0. \tag{3}\label{3}
\end{equation*}
Note also that
\begin{equation*}
\int_{\R}dx\,f(x)e^{ix^2y^2}=\frac a{\sqrt{1-2 i y^2}}
\end{equation*}
and hence
\begin{equation*}
\begin{aligned}
J_f(t)&=
a\int_0^{\sqrt t}\frac{dy}{(\frac y{\sqrt t}-z_1)(\frac y{\sqrt t}-z_2)\sqrt{1-2 i y^2}}.
\end{aligned}
\tag{4}\label{4}
\end{equation*}
For any fixed real $A>0$, by dominated convergence (which holds in view of \eqref{3}),
\begin{equation*}
\int_0^A\frac{dy}{(\frac y{\sqrt t}-z_1)(\frac y{\sqrt t}-z_2)\sqrt{1-2 i y^2}}
\to\frac1{z_1z_2}\int_0^A\frac{dy}{\sqrt{1-2 i y^2}}\ll1
\end{equation*}(as $t\to\infty$).
We write $E\ll F$ to mean $|E|=O(F)$.
So, letting $A$ go to $\infty$ slowly enough, we will have $A=o(\sqrt t)$ and
\begin{equation*}
\int_0^A\frac{dy}{(\frac y{\sqrt t}-z_1)(\frac y{\sqrt t}-z_2)\sqrt{1-2 i y^2}}
=o(\ln t).
\end{equation*}
Also, we will have
\begin{equation*}
\int_A^{\sqrt t}\frac{dy}{(\frac y{\sqrt t}-z_1)(\frac y{\sqrt t}-z_2)\sqrt{1-2 i y^2}} \\
\sim\int_A^{\sqrt t}\frac{dy}{(\frac y{\sqrt t}-z_1)(\frac y{\sqrt t}-z_2)y\sqrt{-2 i}}
\sim\frac{1+i}{4z_1z_2}\,\ln t. \tag{5}\label{5}
\end{equation*}
The latter asymptotic expression in \eqref{5} can be obtained by taking the latter integral in \eqref{5} in closed form, by partial fraction decomposition. It can also be obtained by writing $\int_A^{\sqrt t}=\int_A^{t^b}+\int_{t^b}^{t^{1/2}}$ for $b\in(0,1/2)$ such that $b$ is close to $1/2$.
So, assuming $\psi(0)\ne0$, by \eqref{4},
\begin{equation*}
J_f(t)\sim
a\frac{1+i}{4z_1z_2}\,\ln t
=\frac{1+i}{4z_1z_2}\,\sqrt{2\pi}\,\psi(0)\ln t.
\tag{6}\label{6}
\end{equation*}
Next,
\begin{equation*}
J_h(t)= \int_{\R}dx\,\sgn(x)g(x)K(t,x), \tag{7}\label{7}
\end{equation*}
where
\begin{equation*}
K(t,x):=\int_0^{\sqrt t}\frac{dy\,e^{ix^2y^2}|x|}{(\frac y{\sqrt t}-z_1)(\frac y{\sqrt t}-z_2)}
=\frac12\int_0^{r^2}dv\, e^{iv}H_r(v),
\end{equation*}
\begin{equation*}
r:=|x|\sqrt t,
\end{equation*}
\begin{equation*}
H_r(v):=\frac1{\sqrt v\,(\frac{\sqrt v}r-z_1)(\frac{\sqrt v}r-z_2)}.
\end{equation*}
Further,
\begin{equation*}
2K(t,x)=K_1+K_2, \tag{8}\label{8}
\end{equation*}
where
\begin{equation*}
K_1:=\int_0^{1\wedge r^2}dv\, e^{iv}H_r(v),\quad K_2:=\int_{1\wedge r^2}^{r^2}dv\, e^{iv}H_r(v),
\end{equation*}
$u\wedge w:=\min(u,w)$.
For $v\in(0,1\wedge r^2)$, we have $H_r(v)\ll\frac1{\sqrt v}$ and hence
\begin{equation*}
K_1\ll1. \tag{9}\label{9}
\end{equation*}
Note that $K_2=0$ if $r\le1$. If now $r>1$, then
\begin{equation*}
H'_r(v)=-\frac{H_r(v)}{2v}\,\Big(1+\frac1{\sqrt v-rz_1}+\frac1{\sqrt v-rz_2}\Big)
\ll \frac{|H_r(v)|}v\ll\frac1{v^{3/2}}
\end{equation*}
for $v>0$; so, integrating by parts, we see that
\begin{equation*}
K_2\ll1. \tag{10}\label{10}
\end{equation*}
Collecting \eqref{7}--\eqref{10} and recalling that $g$ is an integrable function, we see that
\begin{equation*}
J_h(t)\ll1. \tag{11}\label{11}
\end{equation*}
Collecting \eqref{2}, \eqref{6}, and \eqref{10}, we conclude that
\begin{equation*}
I(t)\sim
\frac{1+i}{4z_1z_2}\,\sqrt{2\pi}\,\psi(0)\frac{\ln t}{\sqrt t}, \tag{12}\label{12}
\end{equation*}
as $t\to\infty$.
As seen from the above proof, for \eqref{12} to hold it is enough that the function $\R\setminus\{0\}\ni x\mapsto\frac{\psi(x)-\psi(0)}x$ be integrable, with $\psi(0)\ne0$. (Of course, $g$ will be integrable if, as stated by the OP, "$\psi$ is a smooth real-valued function with compact support".)
Concerning the case $\psi(0)=0$: Then the bounds on $H_r(v)$ and $H'_r(v)$ developed above provide for dominated convergence. So, taking into account that for each real $x\ne0$
\begin{equation}
H_r(v)\to\frac1{z_1z_2\sqrt v},
\end{equation}
we have
\begin{equation}
K(t,x)\to\frac1{2z_1z_2}\int_0^\infty\frac{dv\,e^{iv}}{\sqrt v}
=\frac{1+i}{2z_1z_2}\,\sqrt{\frac\pi2}
\end{equation}
and
\begin{equation*}
J_h(t)\to C_\psi:= c_\psi \frac{1+i}{2z_1z_2}\,\sqrt{\frac\pi2},
\end{equation*}
where
\begin{equation}
c_\psi:=\int_{\R}dx\,\sgn(x)g(x)=\int_{\R}dx\,\frac{\psi(x)}{|x|}.
\end{equation}
Therefore and because here $f=0$ and hence $J_f(t)=0$, we conclude that
\begin{equation*}
I(t)\sim \frac{C_\psi}{\sqrt t},
\end{equation*}
as $t\to\infty$, provided that $\psi(0)=0$, $c_\psi\ne0$, and the function $g$ is integrable. (In the case when $\psi(0)=0$, we have $g(x)=\psi(x)/x$ for $x\ne0$. As was already noted, $g$ will be integrable if, as stated by the OP, "$\psi$ is a smooth real-valued function with compact support".)
If $\psi(0)=0$ and $c_\psi=0$, then one has to dig deeper yet, possibly ad infinitum.
However, if $\psi$ is a "smooth bump", as you said in a comment, then apparently $\psi\ge0$ and hence $c_\psi>0$ (unless $\psi=0$ and hence $I(t)=0$).
Best Answer
We can evaluate $I(x)$ explicitly, and then asymptotically.
Indeed, using the substitution $s=ru/x$, we get \begin{equation*} I(x)=\frac1{\sqrt x}\lim_{R\to\infty}J_R(x), \tag{1}\label{1} \end{equation*} where \begin{equation*} \begin{aligned} J_R(x)&:=\int_0^R dr\,e^{ir}\int_0^\infty \frac{du}{\sqrt u\,(u+\sqrt u+1)}e^{-ru/x} \\ &=\int_0^\infty \frac{du}{\sqrt u\,(u+\sqrt u+1)}\int_0^R dr\,e^{(i-u/x)r} \\ &=\int_0^\infty \frac{du}{\sqrt u\,(u+\sqrt u+1)}\frac{1-e^{(i-u/x)R}}{u/x-i}. \end{aligned} \end{equation*} Next, (i) for any real $u,x>0$ we have $\dfrac{1-e^{(i-u/x)R}}{u/x-i}\to\dfrac1{u/x-i}$ as $R\to\infty$, (ii) for any real $u,x,R>0$ we have $\Big|\dfrac{1-e^{(i-u/x)R}}{u/x-i}\Big|\le\dfrac2{|u/x-i|}\le2$, and (iii) for any real $x>0$ we have $\displaystyle{\int_0^\infty \frac{du}{\sqrt u\,(u+\sqrt u+1)}\,2<\infty}$.
So, by dominated convergence, \begin{equation*} \lim_{R\to\infty}J_R(x)=J(x), \tag{2}\label{2} \end{equation*} where \begin{equation*} \begin{aligned} &J(x):=\int_0^\infty \frac{du}{\sqrt u\,(u+\sqrt u+1)}\frac1{u/x-i} \\ &=\frac{x \left(-18 \ln x+\pi \left(8 i \sqrt{3} x+(9-9 i) \sqrt{2} \sqrt{x}-\frac{18 \sqrt[4]{-1}}{\sqrt{x}}+4 \sqrt{3}+9 i\right)\right)}{18 (-1+x (x-i))} \end{aligned} \tag{3}\label{3} \end{equation*} (note that the integrand in \eqref{3} is rational in $\sqrt u\,$, so that one can use partial fraction decomposition to get \eqref{3}). So, as $x\to\infty$, \begin{equation*} J(x)\to c:=\frac{4 i \pi }{3 \sqrt{3}}, \end{equation*} so that, by \eqref{1}, \begin{equation*} I(x)\sim \frac c{\sqrt x}. \end{equation*}