Step 1: Introducing an extra parameter
Define
$$\phi(\alpha)=\int^\frac{\pi}{2}_0\arctan\left(\frac{\sin{x}}{\cos{x}-\frac{1}{\alpha}}\right)\cot{x}\ {\rm d}x$$
Differentiating yields
\begin{align}
\phi'(\alpha)
=&-\int^\frac{\pi}{2}_0\frac{\cos{x}}{1-2\alpha\cos{x}+\alpha^2}{\rm d}x\\
=&\frac{\pi}{4\alpha}-\frac{1+\alpha^2}{2\alpha(1-\alpha^2)}\int^\frac{\pi}{2}_0\frac{1-\alpha^2}{1-2\alpha\cos{x}+\alpha^2}{\rm d}x
\end{align}
Step 2: Evaluation of $\phi'(\alpha)$
For $|\alpha|<1$, the following identity holds.
$$\frac{1-\alpha^2}{1-2\alpha\cos{x}+\alpha^2}=1+2\sum^\infty_{n=1}\alpha^n\cos(nx)$$
Therefore,
\begin{align}
\phi'(\alpha)
=&\frac{\pi}{4\alpha}-\frac{1+\alpha^2}{2\alpha(1-\alpha^2)}\left(\frac{\pi}{2}+2\sum^\infty_{n=0}\frac{(-1)^n}{2n+1}\alpha^{2n+1}\right)\\
=&-\frac{\pi\alpha}{2(1-\alpha^2)}-\frac{\arctan{\alpha}}{\alpha}-\frac{2\alpha\arctan{\alpha}}{1-\alpha^2}\tag1
\end{align}
Step 3: The Closed Form
Integrating back, we get
\begin{align}
&\color{red}{\Large{\phi(\sqrt{2}-1)}}\\
=&\left(\frac{\pi}{4}+\arctan{\alpha}\right)\ln(1-\alpha^2)\Bigg{|}^{\sqrt{2}-1}_0-\int^{\sqrt{2}-1}_0\left[\color{#FF4F00}{\frac{\arctan{\alpha}}{\alpha}}+\color{#00A000}{\frac{\ln(1-\alpha^2)}{1+\alpha^2}}\right]{\rm d}\alpha\tag2\\
=&\frac{3\pi}{8}\ln(2\sqrt{2}-2)\color{blue}{\underbrace{\color{black}{-\int^\frac{\pi}{8}_0\color{#FF4F00}{2x\csc{2x}}\ {\rm d}x+\int^\frac{\pi}{8}_0\color{#00A000}{2\ln(\cos{x})}\ {\rm d}x}}}-\int^\frac{\pi}{8}_0\color{#00A000}{\ln(\cos{2x})}\ {\rm d}x\tag3\\
=&\frac{3\pi}{8}\ln(2\sqrt{2}-2)\color{blue}{-x\ln(\tan{x})\Bigg{|}^\frac{\pi}{8}_0+\int^\frac{\pi}{8}_0\ln\left(\frac{1}{2}\sin{2x}\right)\ {\rm d}x}-\int^\frac{\pi}{8}_0\ln(\cos{2x})\ {\rm d}x\tag4\\
=&\frac{\pi}{4}\ln(2\sqrt{2}-2)+\int^\frac{\pi}{8}_0\ln(\tan{2x})\ {\rm d}x\\
=&\frac{\pi}{4}\ln(2\sqrt{2}-2)-2\sum^\infty_{n=0}\frac{1}{2n+1}\int^\frac{\pi}{8}_0\cos\left((8n+4)x\right)\ {\rm d}x\tag5\\
=&\frac{\pi}{4}\ln(2\sqrt{2}-2)-2\sum^\infty_{n=0}\frac{\cos(n\pi)}{(2n+1)(8n+4)}\\
=&\frac{\pi}{4}\ln(2\sqrt{2}-2)-\frac{1}{2}\sum^\infty_{n=0}\frac{(-1)^n}{(2n+1)^2}\tag6\\
=&\boxed{\color{red}{\Large{\displaystyle\frac{\pi}{4}\ln(2\sqrt{2}-2)-\frac{\mathbf{G}}{2}}}}\tag7
\end{align}
Explanation:
$(2):$ Integrated $(1)$ from $0$ to $\sqrt{2}-1$. Integrated $\dfrac{2\alpha\arctan{\alpha}}{1-\alpha^2}$ by parts.
$(3):$ Applied the substitution $\alpha=\tan{x}$.
$(4):$ Integrated $-2x\csc{2x}$ by parts.
$(5):$ Used the Fourier series of $\ln(\tan{2x})$.
$(6):$ $\cos(n\pi)=(-1)^n$ for $n\in\mathbb{N}$.
$(7):$ Used the definition of Catalan's constant.
It's an interesting integral. I think you are stuck on how to tackle $\theta$ sitting in the $cos(2\sin(\theta) - \theta)\,$. I will use complex integration to solve it.
Put $cos(2\sin(\theta) - \theta)\, = \Re\; e^{i(2\sin(\theta) - \theta)}$. Then the integral $I$ becomes $$I = \Re\int_0^\pi \,e^{2\cos(\theta)}e^{i(2\sin(\theta) - \theta)}\,\mathrm{d}\theta$$
Now I want to convert $I$ into a closed line integral and for that, I try the substitution $$\theta \rightarrow 2\pi-\theta$$ in the original integral. It is easy to see then that integral remains unchanged so, by the property of definite integrals for real variables, $$I = \frac{1}{2}\int_0^{2\pi} e^{2\cos(\theta)}\cos(2\sin(\theta) - \theta)\,d\theta$$
Put $$\int_0^{2\pi} e^{2\cos(\theta)}\cos(2\sin(\theta) - \theta)\,d\theta =\Re\int_0^{2\pi} \,e^{2\cos(\theta)}e^{i(2\sin(\theta) - \theta)}\,\mathrm{d}\theta = J$$
Then $I = J/2$. Now substitute $\cos(\theta) + i \sin(\theta) = z\;$ in $J\;$. Then clearly,
$$\int_0^{2\pi}\,e^{2\cos(\theta)}e^{i(2\sin(\theta) - \theta)}\,\mathrm{d}\theta = -i\oint \limits_{|z|=1}\frac{e^{2z}}{z^2}\mathrm{d}z$$
Now, $0$ is the only point of singularity of the integrand in the complex integral. Further, it is a pole of order $2$ and its residue is given by $$Res(0)= \frac{2e^{2\times0}}{1!} = 2$$
Then, by applying Cauchy's Residue theorem, we get
$$\oint_{|z|=1}\frac{e^{2z}}{z^2}\mathrm{d}z = 2\pi i \times (1 \times 2)= 4\pi i$$
Therefore, $I = J/2 = \frac{\Re(4\pi i \times (-i))}{2} = \frac{\Re(4\pi)}{2} = 2\pi.$
Best Answer
First begin with the substitution $s=\tan\theta\implies \frac{1}{1+s^2}ds=d\theta$, giving us \begin{align*}I&:=\int_{0}^{\frac{\pi}{2}}\int_{0}^{1}e^{t+t^{\tan\theta}}\, dt\, d\theta \\ &\,=\int_{0}^{\infty}\int_{0}^{1}\frac{\exp(t+t^s)}{1+s^2}\, dt\, ds\stackrel{s\to 1/s}{=}\int_{0}^{\infty}\int_{0}^{1}\frac{\exp(t+t^{1/s})}{1+s^2}\, dt\, ds \\ &\!\!\stackrel{t\to t^s}{=}\int_{0}^{\infty}\int_{0}^{1}\frac{\exp(t+t^s)s t^{s-1}}{1+s^2}\, dt\, ds\\ &=\int_{0}^{\infty}\int_{0}^{1}\frac{\exp(t+t^s)(s t^{s-1}+1)}{1+s^2}\, dt\, ds-\int_{0}^{\infty}\int_{0}^{1}\frac{\exp(t+t^s)}{1+s^2}\, dt\, ds \end{align*} Now notice that since $\frac{d}{dt}\exp(t+t^s)=\exp(t+t^s)(st^{s-1}+1)$, then integrating with respect to $t$ on the first integral is trivial, and gives us $$I=\int_{0}^{\infty}\frac{e^2-1}{1+s^2}\, ds-I\implies I=\frac{\pi}{4}(e^2-1),$$ as desired. $\square$