If we set:
$$\alpha= \frac{(ab-1)^2+(a-b)^2}{(ab+1)^2+(a+b)^2}$$
$$\beta= \frac{(ab+1)^2-(a+b)^2}{(ab+1)^2+(a+b)^2}$$
Then it follows that:
$$\int_0^1 \frac{\tanh^{-1} (\beta t) dt}{t\sqrt{(1-t)(1- \alpha t)}}=\log (a) \log (b)$$
I have derived this result in a very roundabout way, most of the details you can see in this post, however from the symmetry of it I suspect there may be better and more clear ways to prove it, which is why I'm asking a separate question.
Aside from the proof, I'm interested in deeper reasons or implications for this identity (if they exist) and some references to similar ones.
Best Answer
We can solve this integral using only substitutions and integration by parts, as follows: $$I:=\int_0^1 \frac{\operatorname{arctanh} (\beta t) }{t\sqrt{(1-t)(1- \alpha t)}}dt=\int_0^1 \frac{\operatorname{arctanh}(\beta t)}{t(1-t)}\sqrt{\frac{1-t}{1-\alpha t}}dt$$ $$\overset{\large \frac{1-t}{1-\alpha t}=x}=\int_0^1 \frac{\operatorname{arctanh}\left(\beta \frac{1-x}{1-\alpha x}\right)}{\sqrt x(1-x)}dx\overset{x=y^2}=2\int_0^1 \frac{\operatorname{arctanh}\left(\beta \frac{1-y^2}{1-\alpha y^2}\right)}{1-y^2}dy$$ $$\overset{\large y=\frac{1-x}{1+x}}=\int_0^1 \operatorname{arctanh}\left( \frac{4\beta x}{(1+x)^2-\alpha (1-x)^2}\right)\frac{dx}{x}=\frac12 \int_0^1 \ln\left(\frac{\left(ab+x\right)\left(\frac{1}{ab}+x\right)}{\left(\frac{a}{b}+x\right)\left(\frac{b}{a}+x\right)}\right)\frac{dx}{x}$$ $$\overset{IBP}=\frac12 \int_0^1 \ln x \left(\frac{1}{\frac{a}{b}+x}+\frac{1}{\frac{b}{a}+x}-\frac{1}{ab+x}-\frac{1}{\frac{1}{ab}+x}\right)dx$$
In each of the integral from above we will simplify the denominator using the substitution $x\to kx$, where $k$ is the constant found in each denominator.
$$\Rightarrow I=\frac12 \left(\int_0^\frac{b}{a}\frac{\ln\left(\frac{a}{b}x\right)}{1+x}dx+\int_0^\frac{a}{b}\frac{\ln\left(\frac{b}{a}x\right)}{1+x}dx-\int_0^\frac{1}{ab}\frac{\ln\left(ab x\right)}{1+x}dx-\int_0^{ab}\frac{\ln\left(\frac{x}{ab}\right)}{1+x}dx\right)$$ $$\small =\color{red}{\frac12} \left(\ln\left(\frac{a}{b}\right)\ln\left(1+\frac{b}{a}\right)+\ln\left(\frac{b}{a}\right)\ln\left(1+\frac{a}{b}\right)-\ln(ab)\ln\left(1+\frac{1}{ab}\right)-\ln\left(\frac{1}{ab}\right)\ln\left(1+ab\right)\right)$$ $$+\color{chocolate}{\frac12}\left(\int_0^\frac{b}{a}\frac{\ln x}{1+x}dx+\int_0^\frac{a}{b}\frac{\ln x}{1+x}dx-\int_0^\frac{1}{ab}\frac{\ln x}{1+x}dx-\int_0^{ab}\frac{\ln x}{1+x}dx\right)$$ We can also rewrite the four integrals from above as: $$\color{blue}{\int_\frac{1}{ab}^\frac{b}{a}\frac{\ln x}{1+x}dx}+\int_{ab}^\frac{a}{b}\frac{\ln x}{1+x}dx\overset{\color{blue}{x\to \frac{1}{x}}}=\color{blue}{\int_{ab}^\frac{a}{b}\frac{\ln x}{x}dx-\int_{ab}^\frac{a}{b}\frac{\ln x}{1+x}dx}+\int_{ab}^\frac{a}{b}\frac{\ln x}{1+x}dx$$ $$=\int_{ab}^\frac{a}{b}\frac{\ln x}{x}dx=\frac{\ln^2 x}{2}\bigg|_{ab}^\frac{a}{b}=-2\ln a\ln b$$ So with some algebra for the first term we finally get: $$I=\color{red}{\frac12}\left(4\ln a \ln b\right)+\color{chocolate}{\frac12}\left(-2\ln a \ln b\right)=\boxed{\ln a\ln b}$$
An alternative approach using Feynman's trick can be found here, which shows: $$\int_0^1 \ln\left(\frac{\left(ab+x\right)\left(\frac{1}{ab}+x\right)}{\left(\frac{a}{b}+x\right)\left(\frac{b}{a}+x\right)}\right)\frac{dx}{x}=2\ln a\ln b$$ It might be useful in the future so I'll also mention that, since $\int_0^1 \frac{\ln x}{t+x}dx=\operatorname{Li}_2\left(-\frac{1}{t}\right)$ the following Dilogarithm identity arises from above: $$\boxed{\operatorname{Li}_2\left(-\frac{a}{b}\right)+\operatorname{Li}_2\left(-\frac{b}{a}\right)-\operatorname{Li}_2\left(-ab\right)-\operatorname{Li}_2\left(-\frac{1}{ab}\right)=2\ln a\ln b;\ a,b>0}$$