Let $t = 1 + y^3$, we can rewrite $B\left(\alpha^2; \frac12, \frac13 \right)$ as
$$\int_0^{\alpha^2} t^{-1/2} (1-t)^{-2/3} dt
= \int_{-1}^{-\sqrt[3]{1-\alpha^2}} \frac{3y^2dy}{\sqrt{1+y^3} y^2}
= 3 \int_{-1}^{-\sqrt[3]{1-\alpha^2}} \frac{dy}{\sqrt{1+y^3}}$$
Following the setup in my answer to a related question. Let
$\;\displaystyle\eta = \frac{\Gamma\left(\frac13\right)\Gamma\left(\frac16\right)}{\sqrt{3\pi}}\;$ and $\wp(z)$ be the Weierstrass elliptic $\wp$ function with fundamental periods $1$ and $e^{i\pi/3}$. $\wp(z)$ is known to satisfy an ODE of the form
$$\wp'(z)^2 = 4 \wp(z)^3 - g_2 \wp(z) - g_3\quad\text{ where }\quad g_2 = 0 \;\text{ and }\;g_3 = \frac{\eta^6}{16}$$
If one perform variable substitution $\;\displaystyle y = -\frac{4}{\eta^2} \wp\left(\frac{iz}{\eta}\right)$, one has
$$\frac{dy}{\sqrt{1+y^3}} = -dz\quad\text{ and }\quad
\begin{cases}
y\left(\frac{\sqrt{3}\eta}{3}\right)
= -\frac{4}{\eta^2}\wp\left(i\frac{\sqrt{3}}{3}\right) = 0\\
\\
y\left(\frac{\sqrt{3}\eta}{2}\right)
= -\frac{4}{\eta^2}\wp\left(i\frac{\sqrt{3}}{2}\right) = -1
\end{cases}$$
Using this, we can express conjecture $(8)$ in terms of $y(\cdot)$ and/or $\wp(\cdot)$:
$$\begin{align}
& B\left(\alpha^2; \frac12, \frac13 \right)
\stackrel{?}{=} \frac{\sqrt{\pi}}{2}\frac{\Gamma\left(\frac13\right)}{\Gamma\left(\frac56\right)} = \frac{\sqrt{3}}{4}\eta\\
\iff & 3\left[y^{-1}(-1) - y^{-1}(-\sqrt[3]{1-\alpha^2})\right]
\stackrel{?}{=} \frac{\sqrt{3}}{4}\eta\\
\iff & y^{-1}(-\sqrt[3]{1-\alpha^2})
\stackrel{?}{=} \frac{5\sqrt{3}}{12}\eta\\
\iff & \frac{4}{\eta^2}\wp\left(i\frac{5\sqrt{3}}{12}\right)
\stackrel{?}{=} \sqrt[3]{1-\alpha^2}
\end{align}
$$
Let $u_0 = i\frac{\sqrt{3}}{3}$, $u_{-1} = i\frac{\sqrt{3}}{2}$ and $u = i\frac{5\sqrt{3}}{12} = \frac12(u_0 + u_{-1})$. Using the addition formula for $\wp$ function,
we have
$$
\wp(2u) = \wp(u_0 + u_{-1})
= \frac14\left[\frac{\wp'(u_0)-\wp'(u_{-1})}{\wp(u_0)-\wp(u_{-1})}\right]^2 - \wp(u_0) - \wp(u_{-1})\\
=\frac14\left[\frac{-i\frac{\eta^3}{4} - 0}{0 - \frac{\eta^2}{4}}\right]^2 - 0 - \frac{\eta^2}{4}
= -\frac{\eta^2}{2}
$$
Using the duplication formula of $\wp$ function, we get
$$
-\frac{\eta^2}{2} = \wp(2u) = \frac14\left(\frac{(6\wp(u)^2-\frac12 g_2)^2}{4\wp(u)^3-g_2\wp(u)-g_3}\right) -2\wp(u)
= \frac{9\wp(u)^4}{4\wp(u)^3-\frac{\eta^4}{16}} - 2\wp(u)
$$
Let $Y = \frac{4}{\eta^2}\wp(u)$ and $A^2 = 1 - Y^3$, above condition is equivalent to
$$\begin{align}
& Y^4 + 8 Y^3 + 8 Y - 8 = 0\tag{*1a}\\
\iff & Y(8+Y^3) = 8(1-Y^3)\tag{*1b}\\
\implies & (9-A^2)^3(1-A^2) = 512A^6\tag{*1c}\\
\iff & (A^4-24A^3+18A^2-27)(A^4+24A^3+18A^2-27) = 0\tag{*1d}
\end{align}$$
- Since $\alpha$ is a root for one of the factors in $(*1d)$, $A = \alpha$ satisfies $(*1d)$ and hence $(*1c)$.
- Since $0 < \alpha < 1$ implies $1 - \alpha^2 > 0$, $(*1c) \implies (*1b)$ in this particular case.
i.e. $Y = \sqrt[3]{1-\alpha^2}$ satisfies $(*1b)$ and hence $(*1a)$.
- Since $u$ lies between $u_0$ and $u_{-1}$, $\frac{4}{\eta^2}\wp(u) > 0$. Using
the fact $(*1a)$ has only one positive root, we find $\frac{4}{\eta^2}\wp(u) = \sqrt[3]{1-\alpha^2}$. i.e. conjecture $(8)$ is true.
$$I:=\int_{0}^{\infty}\frac{\ln{(x)}}{\sqrt{x}\,\sqrt{x+1}\,\sqrt{2x+1}}\mathrm{d}x.$$
After first multiplying and dividing the integrand by 2, substitute $x=\frac{t}{2}$:
$$I=\int_{0}^{\infty}\frac{2\ln{(x)}}{\sqrt{2x}\,\sqrt{2x+2}\,\sqrt{2x+1}}\mathrm{d}x=\int_{0}^{\infty}\frac{\ln{\left(\frac{t}{2}\right)}}{\sqrt{t}\,\sqrt{t+2}\,\sqrt{t+1}}\mathrm{d}t.$$
Next, substituting $t=\frac{1}{u}$ yields:
$$\begin{align}
I
&=-\int_{0}^{\infty}\frac{\ln{(2u)}}{\sqrt{u}\sqrt{u+1}\sqrt{2u+1}}\mathrm{d}u\\
&=-\int_{0}^{\infty}\frac{\ln{(2)}}{\sqrt{u}\sqrt{u+1}\sqrt{2u+1}}\mathrm{d}u-\int_{0}^{\infty}\frac{\ln{(u)}}{\sqrt{u}\sqrt{u+1}\sqrt{2u+1}}\mathrm{d}u\\
&=-\int_{0}^{\infty}\frac{\ln{(2)}}{\sqrt{u}\sqrt{u+1}\sqrt{2u+1}}\mathrm{d}u-I\\
\implies I&=-\frac{\ln{(2)}}{2}\int_{0}^{\infty}\frac{\mathrm{d}x}{\sqrt{x}\sqrt{x+1}\sqrt{2x+1}}.
\end{align}$$
Making the sequence of substitutions $x=\frac{u-1}{2}$, then $u=\frac{1}{t}$, and finally $t=\sqrt{w}$, puts this integral into the form of a beta function:
$$\begin{align}
\int_{0}^{\infty}\frac{\mathrm{d}x}{\sqrt{x}\sqrt{x+1}\sqrt{2x+1}}
&=\int_{1}^{\infty}\frac{\mathrm{d}u}{\sqrt{u-1}\sqrt{u+1}\sqrt{u}}\\
&=\int_{1}^{\infty}\frac{\mathrm{d}u}{\sqrt{u^2-1}\sqrt{u}}\\
&=\int_{1}^{0}\frac{t^{3/2}}{\sqrt{1-t^2}}\frac{(-1)}{t^2}\mathrm{d}t\\
&=\int_{0}^{1}\frac{\mathrm{d}t}{\sqrt{t}\,\sqrt{1-t^2}}\\
&=\frac12\int_{0}^{1}\frac{\mathrm{d}w}{w^{3/4}\,\sqrt{1-w}}\\
&=\frac12\operatorname{B}{\left(\frac14,\frac12\right)}\\
&=\frac12\frac{\Gamma{\left(\frac12\right)}\Gamma{\left(\frac14\right)}}{\Gamma{\left(\frac34\right)}}\\
&=\frac{\pi^{3/2}}{2^{1/2}\Gamma^2{\left(\frac34\right)}}
\end{align}$$
Hence,
$$I=-\frac{\ln{(2)}}{2}\frac{\pi^{3/2}}{2^{1/2}\Gamma^2{\left(\frac34\right)}}=-\frac{\pi^{3/2}\,\ln{(2)}}{2^{3/2}\,\Gamma^2{\left(\frac34\right)}}.~~~\blacksquare$$
Possible Alternative: You could also derive the answer from the complete elliptic integral of the first kind instead of from the beta function by making the substitution $t=z^2$ instead of $t=\sqrt{w}$.
$$\begin{align}
\int_{0}^{\infty}\frac{\mathrm{d}x}{\sqrt{x}\sqrt{x+1}\sqrt{2x+1}}
&=\int_{1}^{\infty}\frac{\mathrm{d}u}{\sqrt{u-1}\sqrt{u+1}\sqrt{u}}\\
&=\int_{1}^{\infty}\frac{\mathrm{d}u}{\sqrt{u^2-1}\sqrt{u}}\\
&=\int_{1}^{0}\frac{t^{3/2}}{\sqrt{1-t^2}}\frac{(-1)}{t^2}\mathrm{d}t\\
&=\int_{0}^{1}\frac{\mathrm{d}t}{\sqrt{t}\,\sqrt{1-t^2}}\\
&=2\int_{0}^{1}\frac{\mathrm{d}z}{\sqrt{1-z^4}}\\
&=2\,K{(-1)}\\
&=\frac{\Gamma^2{\left(\frac14\right)}}{2\sqrt{2\pi}}\\
&=\frac{\pi^{3/2}}{2^{1/2}\Gamma^2{\left(\frac34\right)}}.
\end{align}$$
Best Answer
As suggested by Chen Wang, this integral is related to an integral of the form $$ \int_0^1 \frac{dx}{ \sqrt{1-x} \; x^{2/3} (1-zx)^{1/3} } $$ which appears in this forum's Question 879089 by the same author.
Here's a more direct route than given by Kirill, without conversions between definite integrals and hypergeometric functions. A differential $dx \left/ \left(\sqrt[3]{A+Bx} \, (1-x^2)^{2/3}\right)\right.$ has four poles of fractional order, two at $x = \pm 1$ of order $2/3$, and two at $x = \infty$ and $x = -B/A$ of order $1/3$. Hence the differential is invariant under an involution in the form of a fractional linear transformation that switches $-1 \leftrightarrow +1$ and $\infty \leftrightarrow -B/A$. To exploit this symmetry it is convenient to use a coordinate $u = (cx+1)/(x+c)$: for any $c$ we have $u=\pm 1$ at $x = \pm 1$ and $u = c$ at $x = \infty$, and we choose $c>1$ so that $u=-c$ at $x=-B/A$. Explicitly, this change of variable gives $$ {\large\int}_{-1}^1 \frac{dx}{\sqrt[3]{c^2+1+2cx}\ \left(1-x^2\right)^{\small2/3}} = {\large\int}_{-1}^1 \frac{du}{\sqrt[3]{c^2-u^2}\ \left(1-u^2\right)^{\small2/3}} . $$ By symmetry the second $\int_{-1}^1$ is $2\int_0^1$ of the same integrand, and then the change of variable $u^2 = t$ gives $$ {\large\int}_0^1 \frac{dt}{\sqrt{t} \, (c^2-t)^{\small1/3} (1-t)^{\small2/3}}. $$ Now integrating with respect to $X=1-t$ instead of $t$ gives $$ {\large\int}_0^1 \frac{dX}{\sqrt{1-X} \; X^{2/3} \left(c^2-1+X\right)^{\small1/3}}, $$ and we have reached an integral of the desired form.
In the present case we take $c = \frac12 \! \sqrt 5$. Then $c^2+1 + 2cx = (9 + 4\sqrt{5}x)/4$, so our integral is $4^{1/3}$ times the one with $\sqrt[3]{9 + 4\sqrt{5}x}$ in the denominator. Hence $c^2-1 = 1/4$, so we gain another factor of $4^{1/3}$ and we've reached the integral in the first display with $z=-4$. Vladimir Reshetnikov already noted in a comment to his question that the integral seemed to have an exact value at $z = -4$, and the analysis I gave in my answer there (or Kirill's answer here) leads to the period of a CM elliptic curve with $j$-invariant $146329141248 \sqrt{5} - 327201914880$; this curve is related by $5$-isogeny with a curve of $j$-invariant zero, and thus has a period proportional to a Beta integral.