In Bender & Orszag, at least in my edition (the first, 1978), this is problem 6.73 on p. 313. On p. 308, problem 6.19(a), you are asked how many terms one can get using integration by parts. (The answer is one - the second term blows up at the $t=0$ limit.) Your answer is correct for the leading term.
The idea here is to use steepest descents. I will outline here how to go about setting up the steepest descent contours and setting up the integrals used to derive the asymptotic expansions. Really, this case follows closely Example 2 of Sec. 6.6 in B & O.
Recall that the whole idea behind steepest descent is to find an approximation to the complex integral
$$\int_C dz \, h(z) e^{x \rho(z)} $$
as $x \to \infty$. We do this by deforming $C$ to a steepest descent contour $C'$ along which we may end up getting integrals to which we may apply Laplace's method, e.g., decaying exponentials. So, again, consider the integral of interest:
$$\int_0^{\pi/4} dt \, \cos{x t^2} \, \tan^2{t} = \operatorname{Re} \int_0^{\pi/4} dt \, e^{i x t^2} \, \tan^2{t}$$
Our contour $C$ is just a line segment over the real axis. What would be the steepest descent contour $C'$? It is that over which $\operatorname{Im}{\rho(z)}$ is constant. Letting $z=u+i v$, this would mean that we want a contour over which $i z^2 = -2 u v+i (u^2-v^2) $ is constant.
At $z=0$, $\operatorname{Im}{(i z^2)}=0$, so we can use a contour defined by $u=v$. ($u=-v$ defines a steepest ascent contour, which does not deliver an integral useful for Laplace's method.) Thus, the contour $C_1$ coming from $z=0$ is $z=e^{i \pi/4} t$.
We will discuss a contour $C_3$ coming from the other endpoint at $z=\pi/4$ in a minute. We deform the original integral into a closed contour $\gamma=C_1+C_2-C_3-C$ as follows:
The contour $C_2$ is the top piece and vanishes as we go farther along the contours $C_1$ and $C_3$. Thus, by Cauchy's theorem we have
$$\int_0^{\pi/4} dt \, e^{i x t^2} \tan^2{t} = \int_{C_1} dz \, e^{i x z^2} \tan^2{z} - \int_{C_3} dz \, e^{i x z^2} \tan^2{z} $$
Now,
$$\int_{C_1} dz \, e^{i x z^2} \tan^2{z} = e^{i \pi/4} \int_0^{\infty} dt \, e^{-x t^2} \tan^2{\left ( e^{i \pi/4} t \right )} $$
Now, at $z=\pi/4$, $\operatorname{Im}{(i z^2)}=\pi^2/16$. Thus, along $C_3$, $u^2-v^2=\pi^2/16$, or $u=\sqrt{v^2+\pi^2/16}$ and
$$\rho(z) = i z^2 = i \frac{\pi^2}{16} - 2 v \sqrt{v^2+\frac{\pi^2}{16}} $$
As in the book, let $s= 2 v \sqrt{v^2+\frac{\pi^2}{16}}$ and so on, and we find that
$$\int_{C_3} dz \, e^{i x z^2} \tan^2{z} = i e^{i \pi^2 x/16} \frac{2}{\pi} \int_0^{\infty} ds \, \left (1+i \frac{16 s}{\pi^2} \right )^{-1/2} \tan^2{\left (\frac{\pi^2}{16}+i s \right )^{1/2}} \, e^{-x s}$$
I hope you can see where this is going. We expand separately along each contour for large $x$. At $C_1$, i.e. $t=0$, it should be clear that
$$\begin{align} e^{i \pi/4} \int_0^{\infty} dt \, e^{-x t^2} \tan^2{\left ( e^{i \pi/4} t \right )} &= i e^{i \pi/4} \int_0^{\infty} dt \, e^{-x t^2} [t^2+O(t^3)]\\ &= -i e^{i \pi/4} \frac14 \sqrt{\pi} x^{-3/2} + O(x^{-2}) \end{align}$$
At $C_3$, i.e., $t=\pi/4$, we have
$$\begin{align} i e^{i \pi^2 x/16} \frac{2}{\pi} \int_0^{\infty} ds \, \left (1+i \frac{16 s}{\pi^2} \right )^{-1/2} \tan^2{\left (\frac{\pi^2}{16}+i s \right )^{1/2}} \, e^{-x s} &= i e^{i \pi^2 x/16} \frac{2}{\pi} \int_0^{\infty} ds \, \left [1+O(s)\right ] e^{-x s} \\ &= i e^{i \pi^2 x/16} \frac{2}{\pi x} + O(x^{-2}) \end{align}$$
Finally, subtracting the two contributions and taking the real part, we get the first two terms of the expansion:
$$\int_0^{\pi/4} dt \, \cos{x t^2} \, \tan^2{t} = \frac{2}{\pi x} \sin{\left (\frac{\pi^2}{16} x \right )} - \frac14 \sqrt{\frac{\pi}{2}} x^{-3/2} + O(x^{-2})$$
The nice part is that we can keep adding on terms if we want.
Here is a graph verifying the above asymptotic expansion:
The blue plot is the log-log error between a numerical value of the integral and the first term in the expansion. The red plot is the log-log-error with both terms. Note that the first plot has a supremum that has slope -1.5, and the red has one that has slope -2.0. This indicates that the expansion above is correct.
Best Answer
If $f(z)$ and $g(z)$ are analytic functions with zeros of order $m$ and $n$, respectively, at $z=0$, then:
With this in mind, we can write $e^{\sin z} - e^{\tan z} = e^{\tan z} \bigl( e^{\sin z-\tan z}-1 \bigr) = e^{\tan z} f(g(z))$ where $f(z) = e^z-1$ and $g(z) = \sin z-\tan z$. It's easy to see that ($e^{\tan z}$ doesn't have a zero at $z=0$ and) $f(z)$ has a simple zero at $z=0$; so the order of the zero of $e^{\sin z} - e^{\tan z}$ is the same as the order of the zero of $g(z) = \sin z-\tan z$. (Even here we can avoid taking three derivatives by writing $g(x) = \tan z (\cos z-1)$ and using the first fact above.)