The preliminary substitution $x=u^2$ yields the integral
$$2\int u\sqrt{1+u^3}\mathrm du$$
This is indeed in the form of an elliptic integral, where the cubic under the square root factorizes as $(u+1)(u^2-u+1)$.
For handling integrals like these, one first performs a preliminary Möbius substitution. In this case, we let
$$u=-\frac{-1+\sqrt{(-1)^2-(-1)+1}+(-1-\sqrt{(-1)^2-(-1)+1})v}{1+v}=\frac{2\sqrt{3}}{1+v}-(1+\sqrt{3})$$
to give
$$12\int\frac{(1-\sqrt{3}+(1+\sqrt{3})v)}{(1+v)^5}\sqrt{(1-v^2)\left(2\sqrt{3}-3+(2\sqrt{3}+3\right) v^2)}\mathrm dv$$
We can now use the Jacobian elliptic functions. Letting $v=\mathrm{cn}(w|m)$, $\mathrm dv=-\mathrm{sn}(w|m)\mathrm{dn}(w|m)\mathrm dw$ and using the Pythagorean formula $\mathrm{sn}^2(w|m)+\mathrm{cn}^2(w|m)=1$ yields
$$-12\int\frac{(1-\sqrt{3}+(1+\sqrt{3})\mathrm{cn}(w|m))}{(1+\mathrm{cn}(w|m))^5}\sqrt{2\sqrt{3}-3+(2\sqrt{3}+3)\mathrm{cn}^2(w|m)}\mathrm{sn}^2(w|m)\mathrm{dn}(w|m)\mathrm dw$$
We perform a further application of the Pythagorean formula, and factor out a constant:
$$-24\sqrt[4]{3}\int\frac{(1-\sqrt{3}+(1+\sqrt{3})\mathrm{cn}(w|m))}{(1+\mathrm{cn}(w|m))^5}\sqrt{1-\frac{2+\sqrt{3}}{4}\mathrm{sn}^2(w|m)}\mathrm{sn}^2(w|m)\mathrm{dn}(w|m)\mathrm dw$$
From that, if we let $m=\dfrac{2+\sqrt{3}}{4}$, we can then exploit the identity $\mathrm{dn}^2(w|m)+m\,\mathrm{sn}^2(w|m)=1$:
$$-24\sqrt[4]{3}\int\frac{(1-\sqrt{3}+(1+\sqrt{3})\mathrm{cn}(w|m))}{(1+\mathrm{cn}(w|m))^5}\mathrm{sn}^2(w|m)\mathrm{dn}^2(w|m)\;\mathrm dw$$
We can express everything in terms of $\mathrm{cn}$; using the Pythagorean relations and splitting into partial fractions yields
$$2\sqrt[4]{3}\left(3(5+3\sqrt{3})w-6(13+8\sqrt{3})\varrho_1+48(3+2\sqrt{3})\varrho_2-96(1+\sqrt{3})\varrho_3+48\sqrt{3}\varrho_4\right)$$
where for brevity
$$\varrho_k=\int\frac{\mathrm dw}{(1+\mathrm{cn}(w\mid m))^k}$$
Evaluating $\varrho_k$ is algebraically a rather complicated affair; for brevity, I will instead refer you to formulae 341.52-55 in Byrd and Friedman, where a recursion relation is listed. The required members are:
$$\begin{align*}\varrho_1&=w-\varepsilon\left(w \mid \frac{2+\sqrt{3}}{4}\right)+\frac{\mathrm{sn}\left(w \mid \frac{2+\sqrt{3}}{4}\right)\mathrm{dn}\left(w \mid \frac{2+\sqrt{3}}{4}\right)}{1+\mathrm{cn}\left(w \mid \frac{2+\sqrt{3}}{4}\right)}\\\varrho_2&=\frac13\left((3+\sqrt{3})\varrho_1-\frac12(2+\sqrt{3})w+\frac{\mathrm{sn}\left(w\mid \frac{2+\sqrt{3}}{4}\right)\mathrm{dn}\left(w\mid \frac{2+\sqrt{3}}{4}\right)}{\left(1+\mathrm{cn}\left(w\mid \frac{2+\sqrt{3}}{4}\right)\right)^2}\right)\\\varrho_3&=\frac15\left(2\left(3+\sqrt{3}\right)\varrho_2-\frac32(2+\sqrt{3})\varrho_1+\frac{2+\sqrt{3}}{4}w+\frac{\mathrm{sn}\left(w\mid \frac{2+\sqrt{3}}{4}\right) \mathrm{dn}\left(w\mid \frac{2+\sqrt{3}}{4}\right)}{\left(1+\mathrm{cn}\left(w\mid \frac{2+\sqrt{3}}{4}\right)\right)^3}\right)\\\varrho_4&=\frac17\left(3\left(3+\sqrt{3}\right)\varrho_3-\frac52\left(2+\sqrt{3}\right)\varrho_2+\frac{2+\sqrt{3}}{2}\varrho_1+\frac{\mathrm{sn}\left(w\mid \frac{2+\sqrt{3}}{4}\right) \mathrm{dn}\left(w\mid \frac{2+\sqrt{3}}{4}\right)}{\left(1+\mathrm{cn}\left(w\mid \frac{2+\sqrt{3}}{4}\right)\right)^4}\right)\end{align*}$$
where $\varepsilon(w|m)=E(\mathrm{am}(w|m)|m)$ is the Jacobi epsilon function. After much algebra and tears, we end up with
$$\begin{split}\frac{\sqrt[4]{3}}{7} \left((3-\sqrt{3})w-6\,\varepsilon\left(w \mid \frac{2+\sqrt{3}}{4}\right)+\frac{2\,\mathrm{sn}\left(w\mid \frac{2+\sqrt{3}}{4}\right) \mathrm{dn}\left(w\mid \frac{2+\sqrt{3}}{4}\right)}{\left(1+\mathrm{cn}\left(w\mid \frac{2+\sqrt{3}}{4}\right)\right)^4}\left(3\mathrm{cn}^3\left(w\mid \frac{2+\sqrt{3}}{4}\right)+\right.\right. \\ \left.\left.(21+8\sqrt{3})\mathrm{cn}^2\left(w\mid \frac{2+\sqrt{3}}{4}\right)+(9-8\sqrt{3})\mathrm{cn}\left(w\mid \frac{2+\sqrt{3}}{4}\right)+8 \sqrt{3}-9\right)\right)\end{split}$$
Undoing the substitutions yields
$$\begin{split}\frac2{7\sqrt[4]{3}}\left(\frac{2\sqrt[4]{3}\sqrt{1+x^{3/2}}(3+x(1+\sqrt{3}+\sqrt{x}))}{1+\sqrt{3}+\sqrt{x}}-6\sqrt{3} E\left(\arccos\left(\frac{2\sqrt{3}}{1+\sqrt{3}+\sqrt{x}}-1\right)\mid \frac{2+\sqrt{3}}{4}\right)\right. \\ \left.+3(\sqrt{3}-1)F\left(\arccos\left(\frac{2\sqrt{3}}{1+\sqrt{3}+\sqrt{x}}-1\right)\mid \frac{2+\sqrt{3}}{4}\right)\right)\end{split}$$
You can check yourself that the derivative of this expression is $\sqrt{1+x^\frac32}$.
Best Answer
In the last step, you're forgetting to use the chain rule: $$\int \frac{1}{4x \sqrt{(4x)^2-3^2}}dx=\frac14\int \frac{4}{4x \sqrt{(4x)^2-3^2}}dx=\frac{1}{4}\frac 13 \sec^{-1}\frac{|4x|}{3}+c$$ Hence: $$4\int \frac{1}{4x \sqrt{(4x)^2-3^2}}dx= \frac 13 \sec^{-1}\frac{|4x|}{3}+c$$