The integral can be evaluated to
$$\pi^2\cot\pi a~\csc\pi a$$
I will provide a proof later.
Let
$$J(a)=\int_{0}^{\infty}\frac{\ln x}{x^{a}(x+1)}dx$$
$$I(a)=\int_0^\infty\frac{x^{-a}}{x+1}dx$$
Clearly, $J(a)=-I'(a)$.
Evaluation of $I(a)$ is easier.
Let
$$f(z)=\frac{z^{-a}}{z+1}=\frac{\exp(-a(\ln|z|+i\arg z))}{z+1}\hbox{ where }\arg z\in[0,2\pi).$$
Let $C$ be the keyhole contour centered at the origin, avoiding the branch cut of $z^{-a}$.
By residue theorem,
$$\oint_C f(z)dz=2\pi i\operatorname*{Res}_{z=-1}f(z)=2\pi i(-1)^{-a}=2\pi ie^{-\pi i a}\qquad{(1)}$$
Also,
$$\oint_C =\int_{\text{large circle}}+\int_{\text{small circle}}+\int_{\text{upper real axis}}+\int_{\text{lower real axis}}$$
You can easily prove that the first two integrals tend to zero.
Moreover,
$$\int_{\text{upper real axis}}=\int^\infty_0 f(te^{i0})dt=I(a)$$
$$\begin{align}
\int_{\text{lower real axis}}
&=\int_\infty^0 f(te^{i2\pi})dt \\
&=\int_\infty^0\frac{(te^{2\pi i})^{-a}}{t+1}dt \\
&=-e^{-2\pi i a}\int^\infty_0\frac{t^{-a}}{t+1}dt \\
&=-e^{-2\pi i a}I(a) \\
\end{align}
$$
Back to $(1)$,
$$I(a)-e^{-2\pi i a}I(a)=2\pi ie^{-\pi i a}$$
$$(e^{\pi i a}-e^{-\pi i a})I(a)=2\pi i$$
$$\frac{e^{\pi i a}-e^{-\pi i a}}{2i}I(a)=\pi$$
$$(\sin{\pi a})I(a)=\pi$$
$$I(a)=\pi\csc{\pi a}$$
Therefore, $J(a)=-I'(a)=\pi^2\cot\pi a~\csc\pi a$.
$$\color{red}{\int_{0}^{\infty}\frac{\ln x}{x^{a}(x+1)}dx=\pi^2\cot\pi a~\csc\pi a}$$
Some special values are
$$J(1/6)=2\sqrt3\pi^2, ~J(1/4)=\sqrt2\pi^2, ~J(1/3)=2\pi^2/3, ~J(1/2)=0$$
$J(a)$ satisfies the functional equation
$$J(a)=-J(1-a).$$
The OP said in the comments
I am starting to think that is the case. Without the $i$ the integral is easily transformed into a gamma function giving the result.
Regarding this:
- According to arXiv:hep-th/0306238 p.78 the $i$ is absent, therefore I interpret you question as
How to prove $$\frac{d}{ds}\biggr|_{s\rightarrow 0}\frac{1}{\Gamma(s)}
\int_{0}^{\infty} dt
\frac{t^{s-1}e^{-tx}}{(e^{at}-1) (e^{bt}-1)}\stackrel{a,b\to0^+}{\sim}\frac{x^2}{ab}\left(\frac34-\frac{\log x}{2}\right)\qquad(\star)$$ under zeta-regularization?
(Yes, the OP missed a minus sign in the conjectured form. Also, the mentioned arXiv paper states that the definition is true only under zeta-regularization.)
Without the $i$ the integral is easily transformed into a gamma function giving the result.
This is true if you completely neglect convergence issues:
$$\begin{align}
\frac{1}{\Gamma(s)}
\int_{0}^{\infty} dt
\frac{t^{s-1}e^{-tx}}{(e^{at}-1) (e^{bt}-1)}
&\sim \frac{1}{\Gamma(s)}\int_{0}^{\infty} dt \frac{t^{s-1}e^{-tx}}{at\cdot bt} \\
&= \frac{1}{ab\Gamma(s)}\int_{0}^{\infty} t^{s-3}e^{-tx}dt \\
&= \frac{1}{ab\Gamma(s)}\frac{\Gamma(s-2)}{x^{s-2}} \\
&\stackrel{\frac{d}{ds},s\to0}{=}\frac1{ab}\left(\frac34-\frac{\log x}{2}\right)
\end{align}
$$
However the first and third line is not rigorous.
Below is my attempt to rigorously prove $(\star)$, and it is nearly complete - I only failed to solve a definite integral that numerically equals the expected $\frac34$.
Define $$F(s)\Gamma(s):=\int_{0}^{\infty}
\frac{t^{s-1}e^{-tx}}{(e^{at}-1) (e^{bt}-1)}dt$$
or equivalently $$\alpha\beta x^s F(s)\Gamma(s):=\int_{0}^{\infty}
t^{s-1}e^{-t}\cdot\underbrace{\frac{\alpha\beta}{(e^{\alpha t}-1) (e^{\beta t}-1)}}_{g(t)}dt \qquad (1)$$
where $\alpha=\frac ax,\beta=\frac bx$.
Note that $(\star)$ can be rewritten as
$$\alpha\beta F'(0)\stackrel{\alpha,\beta\to0^+}{\sim}\frac34-\frac{\log x}{2}$$
By series expansion, $$g(t)=\frac1{t^2}-\frac{\alpha+\beta}{2}\frac1t+\left(\frac{\alpha^2+3\alpha\beta+\beta^2}{12}\right)+O(t)$$
Since $s$ is near to $0$, the integral is not integrable at $t=0$. We would like to remove $t^{-2}, t^{-1}, t^0$ terms from $g(t)$ to regularize the integral in $(1)$. Noting that
$$\Gamma(s)\zeta(s)=\int^\infty_0 t^{s-1} e^{-t}\cdot\frac1{1-e^{-t}}dt$$
$$\Gamma(s)\zeta(s-1)=\int^\infty_0 t^{s-1} e^{-t}\cdot\frac1{(1-e^{-t})^2}dt$$
we apply zeta-regularization by subtracting $\frac1{1-e^{-t}},\frac1{(1-e^{-t})^2}$ (which are $\sim t^{-1}$ and $\sim t^{-2}$ respectively) from $g(t)$.
After tedious algebra, we find that
$$H(t):=g(t)-G(t)\in O(t)$$
$$G(t)=\frac1{(1-e^{-t})^2}-\underbrace{\left(1+\frac{\alpha+\beta}2\right)}_{k_1}\frac1{1-e^{-t}}+\underbrace{\left(\frac1{12}+\frac{\alpha+\beta+\alpha\beta}{4}+\frac{\alpha^2+\beta^2}{12}\right)}_{k_2}$$
Then,
$$\begin{align}
\alpha\beta x^s F(s)\Gamma(s)
&=\int^\infty_0 t^{s-1} e^{-t}H(t)dt+\int^\infty_0 t^{s-1} e^{-t}G(t)dt \\
&=\int^\infty_0 t^{s-1} e^{-t}H(t)dt+\Gamma(s)\zeta(s-1)-k_1\Gamma(s)\zeta(s)+k_2\Gamma(s) \\
\alpha\beta F(s)&=\frac1{\Gamma(s)}\int^\infty_0 \left(\frac tx\right)^s \frac{H(t)}{t}e^{-t}dt+[\zeta(s-1)-k_1\zeta(s)+k_2]x^{-s} \\
\end{align}
$$
Differentiating and taking $s\to 0$,
$$
\alpha\beta F'(0)=-(\zeta(-1)-k_1\zeta(0)+k_2)\log x+\zeta'(-1)-k_1\zeta'(0)
+\int^\infty_0 \frac{H(t)}{t}e^{-t}dt
$$
Taking $\alpha,\beta\to 0^+$, we have $k_1\to 1, k_2\to\frac1{12}$, $g(t)\to \frac1{t^2}$,
$$H(t)\to \frac1{t^2}-\frac1{(1-e^{-t})^2}+\frac1{1-e^{-t}}-\frac1{12}$$
Plugging in $\zeta$ values,
$$\begin{align}
\alpha\beta F'(0)&\stackrel{\alpha,\beta\to0^+}{\sim} -\frac{\log x}2
+\zeta'(-1)-\zeta'(0) \\
&\,\,\,\,\qquad +\int^\infty_0 \frac{e^{-t}}{t}\left[\frac1{t^2}-\frac1{(1-e^{-t})^2}+\frac1{1-e^{-t}}-\frac1{12}\right]dt \\
\end{align}
$$
Numerically, the constants sum up to $\frac34$, but I have no idea how the integral can be solved analytically.
EDIT: the integral was solved by our integration master @RandomVariable here, and it is equal to $\frac34-\zeta'(-1)+\zeta'(0)$.
Therefore, we arrived at the desired result
$$\alpha\beta F'(0)\stackrel{\alpha,\beta\to0^+}{\sim} -\frac{\log x}2+\frac34$$
Best Answer
Note that
\begin{align*} \operatorname{Im} \left( \int_{L_1} \frac{\mathrm{d}}{\mathrm{d}s} \log(s(s-1)) \, \mathrm{d}s \right) &= \operatorname{Im} \left( \int_{L_1} \left( \frac{1}{s} + \frac{1}{s-1} \right) \, \mathrm{d}s \right) \\ &= \operatorname{Im} \left( \int_{L_1} \frac{1}{s} \, \mathrm{d}s \right) + \operatorname{Im} \left( \int_{L_1} \frac{1}{s-1} \, \mathrm{d}s \right). \end{align*}
Now we invoke the following well-known fact that
$$ \operatorname{Im}\left( \int_{\gamma} \frac{\mathrm{d}z}{z - a} \right) = \text{[change of argument about $a$ along $\gamma$]} $$
for any piecewise $C^1$ path $\gamma$ not passing through $a$. This gives
\begin{align*} \operatorname{Im} \left( \int_{L_1} \frac{1}{s} \, \mathrm{d}s \right) &= \color{blue}{\arctan\left(2(p+\varepsilon)\right)}, \\ \operatorname{Im} \left( \int_{L_1} \frac{1}{s-1} \, \mathrm{d}s \right) &= \color{red}{-\arctan\left(2(p+\varepsilon)\right)}, \\ \end{align*}
see the picture below as well.
Therefore, we have
$$ \operatorname{Im} \left( \int_{L_1} \frac{\mathrm{d}}{\mathrm{d}s} \log(s(s-1)) \, \mathrm{d}s \right) = 0 $$
and letting $\varepsilon \to 0^+$ still gives $0$.