See this for a similar analysis.
I would rescale the integral as follows:
$$\frac{a}{\sqrt{U_0}} \int_{-1}^1 \frac{dz}{[1+(E/U_0) z^2] \sqrt{1-z^2}}$$
To consider this integral, consider the function
$$f(z) = (1-z)^{-1/2} (1+z)^{-1/2} = e^{-(1/2) \log{(1-z)}} e^{-(1/2) \log{(1+z)}}$$
The contour we choose is a dumbbell contour that encircles the chosen branch cut between $[-1,1]$. Individually, however, the branch cuts were $[1,\infty)$ and $(-\infty,-1]$, respectively. Thus, $\arg{(1-z)} \in [0,2 \pi)$ and $\arg{(1+z)} \in [-\pi,\pi)$. Thus, while we may set $\arg{(1+z)} = 0$ on the contour segments above and below the real line, we must have that $\arg{(1-z)} = 0$ below the real line, and $\arg{(1-z)} = 2 \pi$ above the real line.
Above the real axis, then
$$f(z) = (1-z^2)^{-1/2}$$
Below, however,
$$f(z) = (1-z^2)^{-1/2} e^{-i (1/2) 2 \pi} = - (1-z^2)^{-1/2}$$
so that
$$\oint_C \frac{dz}{[1+(E/U_0) z^2] \sqrt{1-z^2}} = 2 \int_{-1}^1 \frac{dz}{[1+(E/U_0) z^2] \sqrt{1-z^2}}$$
You are correct that the residue at infinity is zero. The residues at $z=\pm i \sqrt{(U_0/E)}$ do not cancel because the factor $(1-z^2)^{-1/2}$ takes on different signs above and below the branch cut. Therefore, the residues add.
Putting this all together, I get the value of your original integral as
$$\sqrt{\frac{m}{2}} \frac{a \pi}{\sqrt{E+U_0}}$$
Integration is completely unnecessary here. More generally, never integrate if you can differentiate. And most of the times, geometric series is more than enough.
$$
\begin{aligned}
\frac{1}{(z+2i)^3}&=\frac{1}{2}\frac{\mathrm d^2}{\mathrm dz^2}\frac{1}{(z-2i)+4i}\\
&=\frac12\frac{\mathrm d^2}{\mathrm dz^2}\left[-\frac i4+\frac{1}{16}(z-2i)+\frac{i}{64}(z-2i)^2+\cdots\right]\\
&=\frac{i}{64}-\frac{3}{256}(z-2i)-\frac{3i}{512}(z-2i)^2+\cdots
\end{aligned}
$$
Therefore,
$$
\begin{aligned}
f(x)&=\frac{1}{(z-2i)^3}\left[\frac{i}{64}-\frac{3}{256}(z-2i)-\frac{3i}{512}(z-2i)^2+\cdots\right]\\
&=\frac{i}{64}(z-2i)^{-3}-\frac{3}{256}(z-2i)^{-2}\color{red}{-\frac{3i}{512}(z-2i)^{-1}}+\cdots
\end{aligned}
$$
Using the formula
$$
\frac{1}{1-z}=\sum_{n\ge0}z^n
$$
you should be able to write down a general formula for $a_n$, using the steps described above. I leave this to you.
Best Answer
You have to do the expansion using the particular branch you're interested in. It's quite possible that a point may be a singularity for one branch but not for another, e.g. $f(z) = \dfrac{1}{\sqrt{z} - 1}$ has a pole at $z=1$ for a branch where $\sqrt{1} = 1$, but not for a branch where $\sqrt{1}=-1$.
I hope you're not trying to find a residue at a branch point: that doesn't exist. You only have a residue when you have an isolated singularity, and a branch point is not an isolated singularity. Thus your second example has no isolated singularities: to make the denominator $0$ you'd need $z=0$, but that's a branch point of the square root.
Your first example $1/(\sqrt{z} + \ln(z))$ will have poles where $\sqrt{z} + \ln(z) = 0$, namely $4 W(\pm 1/2)^2$ where $W$ is a branch of the Lambert W function. One such point (using the "main" branch $W_0$ and $+1/2$) is approximately $0.4948664144$, and requires using the principal branches of $\sqrt{z}$ and $\ln(z)$. If that point is $p_0$, we have $$\sqrt{z} + \ln(z) = \left(\dfrac{1}{2 \sqrt{p_0}} + \dfrac{1}{p_0}\right) (z - p_0) + O\left((z-p_0)^2\right)$$ so the residue at $p_0$ is $$\dfrac{1}{\dfrac{1}{2 \sqrt{p_0}} + \dfrac{1}{p_0}}$$ using the same branch of $\sqrt{p_0}$ that you are using for $\sqrt{p_0} + \ln(p_0) = 0$.