I wanted to evaluate the integral
\begin{align*}
\int_0^\infty \frac{\arctan(x)}{\sqrt{x}(1+x^2)}\,dx=\frac{\pi^2}{4\sqrt{2}}-\frac{\pi \ln(2)}{2\sqrt{2}} \tag{1}
\end{align*}
I thought of using Feynman´s trick by considering the integral
$$
\begin{align*}
I(a)&=\int_0^\infty \frac{\arctan(ax)}{\sqrt{x}(1+x^2)}\,dx \tag{2}
\end{align*}
$$
Differentiating $(2)$ w.r. to $a$ we obtain:
\begin{align*}
I^\prime(a)&=\int_0^\infty \frac{x}{\sqrt{x}(1+x^2)(1+a^2x^2)}\,dx\\
&=\int_0^\infty \frac{x^{1/2}}{(1+x^2)(1+a^2x^2)}\,dx\\
&=\frac{1}{a^2-1}\left(a^2 \int_0^\infty \frac{x^{1/2}}{1+a^2x^2}\,dx-\int_0^\infty \frac{x^{1/2}}{1+x^2}\,dx\right)\\
&=\frac{1}{a^2-1}\left(\sqrt{a} \int_0^\infty \frac{x^{1/2}}{1+x^2}\,dx-\int_0^\infty \frac{x^{1/2}}{1+x^2}\,dx\right)\\
&=\frac{1}{a^2-1}\left(\frac{\sqrt{a}}{2} \int_0^\infty \frac{x^{-1/4}}{1+x}\,dx-\frac12\int_0^\infty \frac{x^{-1/4}}{1+x}\,dx\right)\\
&=\frac{\pi}{\sqrt{2}}\left( \frac{\sqrt{a}}{a^2-1}-\frac{1}{a^2-1}\right)\\
\end{align*}
Integrating back
\begin{align*}
I(a)&=\frac{\pi}{\sqrt{2}}\left( \int\frac{\sqrt{a}}{a^2-1}\,da-\int\frac{1}{a^2-1}\,da\right)\\
&=\frac{\pi}{\sqrt{2}}\left( \int\frac{1}{(1-a)(1+a)}\,da-\int\frac{\sqrt{a}}{(1-a)(1+a)}\,da\right)\\
&=\frac{\pi}{\sqrt{2}}\left(\frac12 \int\frac{da}{1-a}+\frac12 \int\frac{da}{1+a}-\frac12 \int\frac{\sqrt{a}}{1-a}\,da-\frac12 \int\frac{\sqrt{a}}{1+a}\,da\right)\\
&=\frac{\pi}{\sqrt{2}}\left(\frac12\ln\left(\frac{1+a}{1-a} \right)- \int\frac{u^2}{1-u^2}\,du- \int\frac{u^2}{1+u^2}\,du\right)\\
&=\frac{\pi}{\sqrt{2}}\left(\frac12\ln\left(\frac{1+a}{1-a} \right)+ \int\frac{u^2-1+1}{u^2-1}\,du- \int\frac{u^2+1-1}{u^2+1}\,du\right)\\
&=\frac{\pi}{\sqrt{2}}\left(\frac12\ln\left(\frac{1+a}{1-a} \right)- \int\frac{du}{1-u^2}+ \int\frac{du}{u^2+1}\right)\\
&=\frac{\pi}{\sqrt{2}}\left(\frac12\ln\left(\frac{1+a}{1-a} \right)- \frac12\ln\left(\frac{1+u}{1-u} \right)+ \arctan(u)+C\right)\\
&=\frac{\pi}{\sqrt{2}}\left(\frac12\ln\left(\frac{1+a}{1-a} \right)- \frac12\ln\left(\frac{1+\sqrt{a}}{1-\sqrt{a}} \right)+ \arctan\left(\sqrt{a}\right)+C\right)\\
&=\frac{\pi}{\sqrt{2}}\left(\operatorname{arctanh}(a)- \operatorname{arctanh}\left(\sqrt{a} \right)+ \arctan\left(\sqrt{a}\right)+C\right)\\
\end{align*}
Now, if we set $a=0$ in $(2)$ we find that $C=0$. Therefore
\begin{align*}
\int_0^\infty \frac{\arctan(ax)}{\sqrt{x}(1+x^2)}\,dx=\frac{\pi}{\sqrt{2}}\left(\operatorname{arctanh}(a)- \operatorname{arctanh}\left(\sqrt{a} \right)+ \arctan\left(\sqrt{a}\right)\right) \tag{3}
\end{align*}
Supposedly, we should now let $a \to 1$ in $(3)$ to find $(1)$, but for $\lim_{a \to 1} \operatorname{arctanh}(a) \to \infty$. Is there a way to fix this problem so Feynman´s trick is still applicable?
To add a backgroud to the question. I was originally trying to solve the integral $\int_0^{\pi/2}\frac{x}{\sqrt{\tan(x)}}\,dx$. By a obvious change of variable we find that $\int_0^{\pi/2}\frac{x}{\sqrt{\tan(x)}}\,dx=\int_0^\infty \frac{\arctan(x)}{\sqrt{x}(1+x^2)}\,dx$.
Now observe that
\begin{align*}
\int_0^{\pi/2}\frac{x}{\sqrt{\tan(x)}}\,dx&=\frac{\pi}{2}\int_0^{\pi/2}\sqrt{\tan(x)}\,dx-\int_0^{\pi/2}x\sqrt{\tan(x)}\,dx & \left(x \to \frac{\pi}{2}-x \right)\\
&=\frac{\pi}{2}\frac{\pi}{\sqrt{2}}-\int_0^{\pi/2}x\sqrt{\tan(x)}\,dx & ( \text{by beta function})\\
&=\frac{\pi^2}{2\sqrt{2}}-\int_0^\infty \frac{\sqrt{x}\arctan(x)}{1+x^2}\,dx\\
&=\frac{\pi^2}{2\sqrt{2}}-\left(\frac{\pi}{2\sqrt{2}}\ln(2)+\frac{{\pi}^2}{4\sqrt{2}}\right)
\end{align*}
last line result follows from this post
Best Answer
Note that, as $a\to 1$, $$\begin{align} \operatorname{arctanh}(a)- \operatorname{arctanh}\left(\sqrt{a} \right)&= \frac12\ln\left(\frac{1+a}{1-a} \right)- \frac12\ln\left(\frac{1+\sqrt{a}}{1-\sqrt{a}}\right)\\ &=\frac12\ln\left(\frac{1+a}{1-a}\cdot\frac{1-\sqrt{a}}{1+\sqrt{a}} \right)\\&=\frac12\ln\left(\frac{1+a}{(1+\sqrt{a})^2} \right)\to-\frac{\ln\left(2 \right)}{2}. \end{align}$$ Therefore, from your work, we find $$ \int_0^\infty \frac{\arctan(ax)}{\sqrt{x}(1+x^2)}\,dx=\frac{\pi}{\sqrt{2}}\left(\operatorname{arctanh}(a)- \operatorname{arctanh}\left(\sqrt{a} \right)+ \arctan\left(\sqrt{a}\right)\right) \to -\frac{\pi \ln(2)}{2\sqrt{2}}+\frac{\pi^2}{4\sqrt{2}}.$$