Consider a related integral $\displaystyle\;J(a) = \int_0^\infty e^{-ax}\tan^{-1}(x) dx\;$.
We can integrate it by part to get:
$$J(a) = \frac{1}{a}\int_0^\infty e^{-ax}\frac{dx}{1+x^2}$$
Using this, we can rewrite our integral as
$$I(a)
= \int_0^\infty \left(\sum_{n=1}^\infty e^{-nax}\right)\tan^{-1}(x)dx
= \sum_{n=1}^\infty J(na)
= \sum_{n=1}^\infty \frac{1}{na}\int_0^\infty e^{-nax}\frac{dx}{1+x^2}$$
Replace $x$ by $x/n$ in each terms, this can be transformed as
$$I(a) = \frac{1}{a}\int_0^\infty e^{-at}\left(\sum_{n=1}^\infty \frac{1}{t^2+n^2}\right) dt
$$
Start with a infinite product expansion of $\displaystyle\;\frac{\sinh(\pi x)}{\pi x}\;$. By taking logarithm and differentiate,
$$\frac{\sinh(\pi x)}{\pi x} = \prod_{n=1}^\infty \left(1 + \frac{x^2}{n^2}\right)
\quad\implies\quad
\sum_{n=1}^\infty \frac{1}{x^2 + n ^2 } = \frac{1}{2x}\left( \pi\coth(\pi x) - \frac{1}{x}\right)
$$
we can rewrite the sum inside above integrand of $I(a)$ and obtain
$$I(a) = \frac{\pi}{2a}F\left(\frac{a}{\pi}\right)
\quad\text{ where }\quad
F(x) = \int_0^\infty e^{-xt} \left(\coth(t) - \frac{1}{t}\right) \frac{dt}{t}
$$
Notice
$$-\frac{dF(x)}{dx} = \int_0^\infty e^{-xt}\left(\coth(t) - \frac{1}{t}\right)dt$$
and compare this with a integral representation of digamma function
$\displaystyle\;\psi(x) = \frac{d\log\Gamma(x)}{dx}\;$,
$$\psi(x) = \int_0^\infty\left(\frac{e^{-t}}{t} - \frac{e^{-xt}}{1-e^{-t}}\right)dt
\quad\implies\quad
\psi\left(\frac{x}{2}\right) =
\int_0^\infty\left(\frac{e^{-2t}}{t} - \frac{2e^{-xt}}{1-e^{-2t}}\right)dt
$$
We get
$$\begin{array}{rrl}
&-\frac{dF(x)}{dx}
&= \int_0^\infty \frac{e^{-2t} - e^{-xt}}{t}dt - \frac12
\left[\psi\left(\frac{x}{2}\right) + \psi\left(\frac{x}{2}+1\right)\right]\\
&&= \log\frac{x}{2} - \frac12\left[\psi\left(\frac{x}{2}\right) + \psi\left(\frac{x}{2}+1\right)\right]\\
\implies &
F(x) &= \text{const.} + \log\Gamma\left(\frac{x}{2}\right) + \log\Gamma\left(\frac{x}{2}+1 \right) - x\log\left(\frac{x}{2e}\right)
\end{array}$$
Since $\lim\limits_{x\to\infty}F(x) = 0$, we can use Stirling's approximation to fix the constant in $F(x)$ to $-\log(2\pi)$. As a result, We find
$$I(a) = \frac{\pi}{2a}\left[ 2\log\Gamma\left(\frac{a}{2\pi}+1\right)
- \log(a)\right] -\frac12\log\left(\frac{a}{2\pi e}\right)
$$
\begin{align}
\int_0^{\Large\frac{\pi}{4}} \sqrt{\tan x} \sqrt{1-\tan x}\,\,dx&=\int_0^1\frac{\sqrt{y(1-y)}}{1+y^2}\,dy\quad\Rightarrow\quad y=\tan x\\
&=\int_0^\infty\frac{\sqrt{t}}{(1+t)(1+2t+2t^2)}\,dt\quad\Rightarrow\quad t=\frac{y}{1-y}\\
&=\int_0^\infty\frac{2z^2}{(1+z^2)(1+2z^2+2z^4)}\,dz\quad\Rightarrow\quad z^2=t\\
&=2\int_0^\infty\left[\frac{2z^2}{1+2z^2+2z^4}+\frac{1}{1+2z^2+2z^4}-\frac{1}{1+z^2}\right]\,dz\\
&=\int_{-\infty}^\infty\left[\frac{2z^2}{1+2z^2+2z^4}+\frac{1}{1+2z^2+2z^4}-\frac{1}{1+z^2}\right]\,dz\\
&=I_1+I_2-\pi
\end{align}
\begin{align}
I_1
&=\int_{-\infty}^\infty\frac{2z^2}{1+2z^2+2z^4}\,dz\\
&=\int_{-\infty}^\infty\frac{1}{z^2+\frac{1}{2z^2}+1}\,dz\\
&=\int_{-\infty}^\infty\frac{1}{\left(z-\frac{1}{\sqrt{2}z}\right)^2+1+\sqrt{2}}\,dz\\
&=\int_{-\infty}^\infty\frac{1}{z^2+1+\sqrt{2}}\,dz\\
&=\frac{\pi}{\sqrt{1+\sqrt{2}}}
\end{align}
where the 4th line we use identity
\begin{align}
\int_{-\infty}^\infty f\left(x\right)\,dx=\int_{-\infty}^\infty f\left(x-\frac{a}{x}\right)\,dx\qquad,\qquad\text{for }\, a>0.
\end{align}
The proof can be seen in my answer here. $I_2$ can be proved in similar manner (see user111187's answer).
\begin{equation}
I_2=\frac{1}{2}\int_{-\infty}^\infty\frac{1}{z^4+z^2+\frac{1}{2}}\,dz=\pi\sqrt{\frac{\sqrt{2}-1}{2}}
\end{equation}
Combine all the results together, we finally get
\begin{equation}
\int_0^{\Large\frac{\pi}{4}} \sqrt{\tan x} \sqrt{1-\tan x}\,\,dx=\frac{\pi}{\sqrt[4]{2}}\sqrt{\frac{2+\sqrt{2}}{2}}-\pi
\end{equation}
Best Answer
$$I=\int_0^{\pi/2}\frac{1+\tanh(x)}{1+\tan(x)}dx=\frac\pi 4+\int_0^{\pi/2}\frac{\tanh(x)}{1+\tan(x)}dx$$
$$I_1=\int_0^{\pi/2}\frac{\tanh(x)}{1+\tan(x)}dx$$
$$\tanh(x)=\frac{\sinh(x)}{\cosh(x)}=\frac{e^x-e^{-x}}{e^x+e^{-x}}=1-2\frac{e^{-x}}{e^x+e^{-x}}$$ so: $$I_1=\frac{\pi}{4}-2\int_0^{\pi/2}\frac{1}{1+\tan(x)}\frac{e^{-x}}{e^x+e^{-x}}dx$$ $$I_2=\int_0^{\pi/2}\frac{1}{1+\tan(x)}\frac{e^{-x}}{e^x+e^{-x}}dx$$
$$\frac{e^{-x}}{e^x+e^{-x}}=\frac{e^{-2x}}{1-(-e^{-2x})}=e^{-2x}\sum_{n=0}^\infty(-1)^ne^{-2nx}=\sum_{n=0}^\infty(-1)^ne^{-2(n+1)x}$$ and so: $$I_2=\int_0^{\pi/2}\frac{1}{1+\tan(x)}\sum_{n=0}^\infty(-1)^ne^{-2(n+1)x}dx=\sum_{n=0}^\infty(-1)^n\int_0^{\pi/2}\frac{e^{-2(n+1)x}}{1+\tan(x)}dx$$ as for this integral its quite messy and I'm not sure what to do from here, It would be easier for $0$ to $\pi/4$ I think. I will say that as $n$ increases the terms get smaller very quickly so an approximation of the first few would be quite accurate if possible.
One possible way I have noticed is that: $$e^{-2.5(n+1)x}\le\frac{e^{-2(n+1)x}}{1+\tan(x)}\le e^{-2.4(n+1)x}$$ so if: $$J(a)=\int_0^{\pi/2}e^{-ax}dx=\frac{1-e^{-a\pi/2}}{a}$$ so we have: $$\sum_{n=0}^\infty(-1)^n\frac{1-e^{-2.5(n+1)\pi/2}}{2.5(n+1)}\le I_2\le \sum_{n=0}^\infty(-1)^n\frac{1-e^{-2.4(n+1)\pi/2}}{2.4(n+1)}$$ and according to wolfram alpha these sums converge and arent too ugly, and we know that: $$I=\pi/2-2I_2$$ thats the best I can do at the moment I'll take another look sometime. It is also worth noting that: $$\sum_{n=0}^\infty\frac{(-1)^n}{n+1}=\ln(2)$$ and the second part of the sum could be expanded into a double summation
Back to add a little to this answer, so far we know: $$\frac{\ln2}{2.5}+\frac 1 {2.5}\sum_{n=1}^\infty \frac 1ne^{-2.5n\pi/2}\le I_2\le \frac{\ln2}{2.4}+\frac 1 {2.4}\sum_{n=1}^\infty \frac 1ne^{-2.4n\pi/2}$$ we will try and focus on sums of the form: $$S(\alpha)=\sum_{n=1}^\infty\frac{\exp(-\alpha n)}{n}=-\ln(e^{-\alpha}(e^\alpha-1))$$ according to wolfram alpha, which we are able to simplify to: $$S(\alpha)=\alpha-\ln(e^\alpha-1)$$ $$\frac{S(\alpha)}{\alpha}=1-\frac{\ln(e^\alpha-1)}{\alpha}$$