Update: Finally, a complete solution. Sorry it took so long.
Split the integral up into 3.
\begin{align}
I
&=-\int^{\sqrt{2}}_1\frac{\log{x}}{x}dx+\int^{\sqrt{2}}_1\frac{\log{((x^2-1)^2+1)}}{x}dx-\int^{\sqrt{2}}_1\frac{\log{((x-1)^2+1)}}{x}dx\\
&=-\frac{1}{8}(\log{2})^2+\frac{1}{2}\int^1_0\frac{\log(1+x^2)}{1+x}dx-\int^{\sqrt{2}-1}_0\frac{\log(1+x^2)}{1+x}dx
\end{align}
The second integral is rather easy to evaluate.
\begin{align}
\frac{1}{2}\int^1_0\frac{\log(1+x^2)}{1+x}dx
&=\frac{1}{2}\int^1_0\int^1_0\frac{x^2}{(1+x)(1+ax^2)}dx \ da\tag1\\
&=\frac{1}{2}\int^1_0\frac{1}{1+a}\int^1_0\frac{1}{1+x}+\frac{x-1}{1+ax^2}dx \ da\\
&=\frac{1}{2}\int^1_0\frac{\log{2}}{1+a}+\frac{\log(1+a)}{2a(1+a)}-\underbrace{\frac{\arctan(\sqrt{a})}{\sqrt{a}(1+a)}}_{\text{Let} \ y=\arctan{\sqrt{a}}}da\\
&=\frac{1}{2}\left[(\log{2})^2+\frac{1}{2}\underbrace{\int^1_0\frac{\log(1+a)}{a}da}_{-\operatorname{Li}_2(-1)=\frac{\pi^2}{12}}-\frac{1}{2}\underbrace{\int^1_0\frac{\log(1+a)}{1+a}da}_{\frac{1}{2}(\log{2})^2}-\frac{\pi^2}{16}\right]\\
&=\frac{3}{8}(\log{2})^2-\frac{\pi^2}{96}
\end{align}
The third integral can be evaluated using dilogarithms.
\begin{align}
\int^{\sqrt{2}-1}_0\frac{\log(1+x^2)}{1+x}dx
&=\sum_{r=\pm i}\int^{\sqrt{2}-1}_0\frac{\log(r+x)}{1+x}dx\tag2\\
&=-\sum_{r=\pm i}\int^{\frac{\lambda}{\sqrt{2}}}_{\lambda}\log\left(r-1+\frac{\lambda}{y}\right)\frac{dy}{y}\tag3\\
&=-\sum_{r=\pm i}\int^{\frac{r-1}{\sqrt{2}}}_{r-1}\frac{\log(1+y)}{y}-\frac{1}{y}\log\left(\frac{y}{r-1}\right)dy\tag4\\
&=\frac{1}{4}(\log{2})^2+\sum_{r=\pm i}\mathrm{Li}_2\left(\frac{1-r}{\sqrt{2}}\right)-\mathrm{Li}_2(1-r)\tag5\\
&=\frac{1}{4}(\log{2})^2+\mathrm{Li}_2(e^{i\pi/4})+\mathrm{Li}_2(e^{-i\pi/4})-\mathrm{Li}_2(\sqrt{2}e^{i\pi/4})-\mathrm{Li}_2(\sqrt{2}e^{-i\pi/4})\\
&=\frac{1}{4}(\log{2})^2-\frac{\pi^2}{96}\tag6\\
\end{align}
It follows that
$$I=-\frac{1}{8}(\log{2})^2+\frac{3}{8}(\log{2})^2-\frac{\pi^2}{96}-\frac{1}{4}(\log{2})^2+\frac{\pi^2}{96}=0$$
Explanation
$(1)$: Differentiate under the integral sign
$(2)$: Factorise $1+x^2$, let $r=\pm i$
$(3)$: Let $\displaystyle y=\frac{\lambda}{1+x}$
$(4)$: Let $\lambda=r-1$
$(5)$: Recognise that $\displaystyle\int\frac{\ln(1+y)}{y}dy=-\mathrm{Li}_2(-y)+C$ and $\displaystyle\int\frac{\ln(ay)}{y}dy=\frac{1}{2}\ln^2(ay)+C$
$(6)$: Use the identities here
$\newcommand{\+}{^{\dagger}}
\newcommand{\angles}[1]{\left\langle\, #1 \,\right\rangle}
\newcommand{\braces}[1]{\left\lbrace\, #1 \,\right\rbrace}
\newcommand{\bracks}[1]{\left\lbrack\, #1 \,\right\rbrack}
\newcommand{\ceil}[1]{\,\left\lceil\, #1 \,\right\rceil\,}
\newcommand{\dd}{{\rm d}}
\newcommand{\down}{\downarrow}
\newcommand{\ds}[1]{\displaystyle{#1}}
\newcommand{\expo}[1]{\,{\rm e}^{#1}\,}
\newcommand{\fermi}{\,{\rm f}}
\newcommand{\floor}[1]{\,\left\lfloor #1 \right\rfloor\,}
\newcommand{\half}{{1 \over 2}}
\newcommand{\ic}{{\rm i}}
\newcommand{\iff}{\Longleftrightarrow}
\newcommand{\imp}{\Longrightarrow}
\newcommand{\isdiv}{\,\left.\right\vert\,}
\newcommand{\ket}[1]{\left\vert #1\right\rangle}
\newcommand{\ol}[1]{\overline{#1}}
\newcommand{\pars}[1]{\left(\, #1 \,\right)}
\newcommand{\partiald}[3][]{\frac{\partial^{#1} #2}{\partial #3^{#1}}}
\newcommand{\pp}{{\cal P}}
\newcommand{\root}[2][]{\,\sqrt[#1]{\vphantom{\large A}\,#2\,}\,}
\newcommand{\sech}{\,{\rm sech}}
\newcommand{\sgn}{\,{\rm sgn}}
\newcommand{\totald}[3][]{\frac{{\rm d}^{#1} #2}{{\rm d} #3^{#1}}}
\newcommand{\ul}[1]{\underline{#1}}
\newcommand{\verts}[1]{\left\vert\, #1 \,\right\vert}
\newcommand{\wt}[1]{\widetilde{#1}}$
With $\ds{r \equiv {1 + \root{3}\ic \over 2} = \expo{\pi\ic/3}}$
\begin{align}&\color{#c00000}{
\int_{0}^{1}{\ln\pars{\ln\pars{1/x}} \over x^{2} - x + 1}\,\dd x}
=\int_{0}^{1}{\ln\pars{\ln\pars{1/x}} \over \pars{x - r}\pars{x - r^{*}}}\,\dd x
\\[3mm]&=\int_{0}^{1}\ln\pars{\ln\pars{1/x}}
\pars{{1 \over x - r} - {1 \over x - r^{*}}}\,{1 \over r - r^{*}}\,\dd x
\\[3mm] & =
{1 \over \Im\pars{r}}\,\Im\int_{0}^{1}
{\ln\pars{\ln\pars{1/x}} \over x - r}\,\dd x
\end{align}
With $\ds{x \equiv \expo{-t}}$:
\begin{align}&\color{#c00000}{
\int_{0}^{1}{\ln\pars{\ln\pars{1/x}} \over x^{2} - x + 1}\,\dd x}
={2\root{3} \over 3}\Im\int_{\infty}^{0}{\ln\pars{t} \over \expo{-t} - r}
\,\pars{-\expo{-t}\,\dd t}
\\[3mm]&=-\,{2\root{3} \over 3}\Im\bracks{{1 \over r}\int_{0}^{\infty}
{\ln\pars{t}\expo{-t} \over 1 - \expo{-t}/r}\,\dd t}
\\[3mm]&=-\,{2\root{3} \over 3}\Im\bracks{{1 \over r}
\sum_{n = 1}^{\infty}{1 \over r^{n - 1}}\int_{0}^{\infty}
\ln\pars{t}\expo{-nt}\,\dd t}
\\[3mm]&=-\,{2\root{3} \over 3}\Im\lim_{\mu\ \to\ 0}\partiald{}{\mu}\bracks{
\sum_{n = 1}^{\infty}r^{-n}\int_{0}^{\infty}t^{\mu}\expo{-nt}\,\dd t}
\\[3mm]&=-\,{2\root{3} \over 3}\Im\lim_{\mu\ \to\ 0}\partiald{}{\mu}\bracks{
\sum_{n = 1}^{\infty}{r^{-n} \over n^{\mu + 1}}\Gamma\pars{\mu + 1}}
\\[3mm]&=-\,{2\root{3} \over 3}\Im\lim_{\mu\ \to\ 0}\partiald{}{\mu}\bracks{
\Gamma\pars{\mu + 1}{\rm Li}_{\mu + 1}\pars{r^{*}}}
\\[3mm]&=-\,{2\root{3} \over 3}\Im\lim_{\mu\ \to\ 0}\partiald{}{\mu}\bracks{
\Gamma\pars{\mu + 1}{\rm Li}_{\mu + 1}\pars{\expo{-\pi\ic/3}}}
\end{align}
$\ds{{\rm Li}_{1}\pars{z} = -\ln\pars{1 - z}}$. Derivatives of the
PolyLogarithm, respect of the order, can be evaluated from its integral representation.
Also, see Hurwitz Zeta Function.
Best Answer
First, we set $x=\cos\theta\ \color{red}{\Rightarrow}\ dx=-\sin\theta\ d\theta$, then \begin{align} \mathcal{I}=\int_0^1\frac{\ln x}{1+x}\arccos(x)\ dx=-\int_0^{\Large\frac{\pi}2}\frac{\sin\theta}{1+\cos\theta}\cdot\theta\ \ln (\cos\theta)\ d\theta.\tag1 \end{align} Using the fact that \begin{align} \tan\left(\frac\theta2\right)=\frac{\sin\theta}{1+\cos\theta} \end{align} and setting $y=\frac\theta2$, then $(1)$ turns out to be \begin{align} \mathcal{I}=-\int_0^{\Large\frac{\pi}2}\theta\tan\left(\frac\theta2\right)\ln (\cos\theta)\ d\theta=-4\int_0^{\Large\frac{\pi}4}y\tan\left(y\right)\ln (\cos2y)\ dy.\tag2 \end{align} Now, setting $t=\tan y\ \color{red}{\Rightarrow}\ dy=\dfrac{dt}{1+t^2}$ yields \begin{align} \mathcal{I}&=-4\int_0^1 t\arctan(t)\ln \left(\frac{1-t^2}{1+t^2}\right)\ \frac{dt}{1+t^2}\\ &=4\int_0^1\left[ \frac{t\arctan(t)}{1+t^2}\ln \left(1+t^2\right)- \frac{t\arctan(t)}{1+t^2}\ln \left(1-t^2\right)\right]\ dt.\tag3 \end{align} Actually, the second integral in $(3)$ has been evaluated by Kirill and it is equal to \begin{align} \int_0^1 \frac{t\arctan(t)}{1+t^2}\ln \left(1-t^2\right)\ dt=-\frac{\pi^3}{48}-\frac{\pi}{8}\log^2 2 +\text{G}\log 2,\tag4 \end{align} where $\text{G}$ is is the Catalan's Constant. We may also refer to the following technique to evaluate \begin{align} \int_0^1 \frac{\ln \left(1+at\right)\ln \left(1+bt\right)}{1+ct}\ dt. \end{align} The first integral in $(3)$ can be evaluated by setting $t=\tan x$, we have \begin{align} \int_0^1 \frac{t\arctan(t)}{1+t^2}\ln \left(1+t^2\right)\ dt=-2\int_0^{\Large\frac{\pi}4}x\tan x\ln (\cos x)\ dx. \end{align} Now, applying IBP by setting $u=x$ and $$ v=\int\tan x\ln (\cos x)\ dx=-\ln^2(\cos x)\quad \color{red}{\Rightarrow}\quad \text{set}\ z= \cos x. $$ Hence \begin{align} \int_0^1 \frac{t\arctan(t)}{1+t^2}\ln \left(1+t^2\right)\ dt&=2x\ln^2(\cos x)\bigg|_{x=0}^{\Large\frac{\pi}4}-2\int_0^{\Large\frac{\pi}4}\ln^2 (\cos x)\ dx\\ &=\frac{\pi}8\ln^22-2\int_0^{\Large\frac{\pi}4}\ln^2 (\cos x)\ dx.\tag5 \end{align} Random Variable has successfully proven that $$ \int_0^{\Large\frac{\pi}4}\ln^2 (\sin x)\ dx=\frac{\pi^{3}}{192} + \frac{\text{G}}{2}\ln2 + \frac{3 \pi}{16} \ln^{2}2 - \Im \ \text{Li}_{3}(1-i)\tag6 $$ and Lucian has shown that $$ \int_0^{\Large\frac{\pi}4}\ln^2 (\sin x)\ dx+\int_0^{\Large\frac{\pi}4}\ln^2 (\cos x)\ dx=\frac{\pi^3}{24}+\dfrac\pi2\ln^22.\tag7 $$ Then by using $(3)$, $(4)$, $(5)$, $(6)$, and $(7)$, we finally obtain
The above result is exactly similar to Cleo's answer. It can be proven by using the following trilogarithm identity
$\displaystyle \operatorname{Li}_3(z)+\operatorname{Li}_3(1-z)+\operatorname{Li}_3\left(1-\frac1z\right)=\zeta(3)+\frac16\ln^3z+\zeta(2)\ln z-\frac12\ln^2z\ln(1-z). $$\tag8$
Setting $z=i$ to $(8)$, Wolfram Alpha shows that $\displaystyle\Im\bigg[\operatorname{Li}_3(1-i)\bigg]=-\Im\bigg[\operatorname{Li}_3(1+i)\bigg]$ since $$ \Im\bigg[\zeta(3)+\frac16\ln^3(i)+\zeta(2)\ln(i)-\frac12\ln^2(i)\ln(1-i)-\operatorname{Li}_3(i)\bigg]=0. $$ Thus, the another closed-form of $\mathcal{I}$ is