This is a technique to solve the more general problem of counting the number of zeros of a polynomial inside the unit circle. One could use it for other curves other than the circle. All is needed is to be able to map it to a line by a rational function.
The idea is to use the argument principle instead: The number of zeros of a polynomial lying inside a loop is the number of times the image of that loop by the polynomial winds around the origin. But the unit circle is hard on additions. That is why a pretty proof by Rouche's can be tricky sometimes.
Let's instead map the unit circle to the imaginary line.
You might know a rational function that does the map, but we can derive it step by step.
- Translate the circle one unit to the right. $z= x-1$.
- Then we do inversion. Inversion would be $x = 1/\overline{y}$. But since the coefficients are real the conjugate won't matter. So we do $x=1/y$. We get a rational function of which we only care about the numerator (a polynomial). If zero is a solution, then $-1$ was a solution of the original polynomial and that we should've tested before hand. After this the circle got mapped to the vertical line passing through (1/2,0).
- Finally we translate to the left by 1/2. y = w+1/2.
So, we get some polynomial with real coefficients. Let's evaluate it at $w = ir$ with $r$ real.
Now, separate imaginary part and real part. Both a polynomials of smaller degree. For this particular problem I think we get
$$(880r^4-392r^2+23)+r(96r^4-912r^2+54)i$$
Now, to determine the number of times this winds around the origin we just need to see how it jumps from quadrant to quadrant. The counting of roots (no need of precise determination) can be done with Sturm's theorem in general.
For this particular problem the work is much easier. For $r=0$ we are at the point (23,0). The polynomials $880r^4-392r^2+23$ and $96r^4-912r^2+54$ are just quadratics in disguise. One can compute the roots if so inclined.
But all it matters is their relative position, which I think it is $ABBAABBA$, where the $A$'s represent roots from the second polynomial and the $B$'s represent the roots of the first one. Take into account the factor $r$ in the imaginary part which also changes its sign when $r$ crosses zero.
That order of the roots tells you the sign of the imaginary part and real part on each of the intervals between the roots. This tells you to which quadrant the whole expression is moving. From the succession of quadrants you count the winding number and that is your number.
We can construct a proof based on homotopy invariance, borrowing ideas from the proof of Rouche's theorem.
Let $\gamma : [0, 1] \to\mathbb C$ parametrise the semi-circular contour:
$$ \gamma(t) = \begin{cases} -1 + 4t & t \in [0, \tfrac 1 2] \\ e^{2\pi i (t - \tfrac 1 2 )} & t \in [\tfrac 1 2 , 1]\end{cases}$$
The number of zeroes of $f$ in the upper half disk is equal to the winding number around the origin for the curve $f \circ \gamma : [0,1] \to \mathbb C^\star$:
$$ f \circ \gamma(t) = \begin{cases} (-1 + 4t)^6 + 3(-1+4t)^4+1 & t \in [0, \tfrac 1 2] \\ e^{12\pi i (t - \tfrac 1 2 )} + 3e^{8\pi i (t - \tfrac 1 2 )} + 1 & t \in [\tfrac 1 2 , 1]\end{cases}$$
As you say, this winding number is hard to evaluate. However, since the $3e^{8\pi i (t - \tfrac 1 2 )}$ term is "dominant" for $t \in [\tfrac 1 2 , 1] $, we would expect the winding number of $f \circ \gamma$ around the origin to be the same as the winding number of the simpler-looking curve $g : [0, 1] \to \mathbb C^\star$, which is defined as:
$$ g(t) := \begin{cases} 3 & t\in [0, \tfrac 1 2] \\ 3e^{8\pi i (t - \tfrac 1 2 )} & t \in [\tfrac 1 2 , 1]\end{cases}.$$
To make this intuition rigorous, we exhibit a homotopy $F: [0,1] \times [0,1] \to \mathbb C^\star$ between $g$ and $f \circ \gamma$. A possible homotopy is
$$ F(s , t) = \begin{cases} 3(1-s) + \left((-1 + 4t)^6 + 3(-1+4t)^4+1\right)s & t \in [0, \tfrac 1 2] \\ se^{12\pi i (t - \tfrac 1 2 )} + 3e^{8\pi i (t - \tfrac 1 2 )} + s & t \in [\tfrac 1 2 , 1]\end{cases}$$
The key thing we need to check is that this homotopy avoids the origin, i.e. $F(s, t) \neq 0$ for all $s$ and $t$:
If $t \in [0, \tfrac 1 2 ]$, then $F(s, t) \neq 0$ for all $s \in [0,1]$, because $3$ and $(-1 + 4t)^6 + 3(-1 + 4t)^4 + 1$ are both strictly positive real numbers.
If $t \in [\tfrac 1 2 , 1]$, then $F(s, t) \neq 0$ for all $s \in [0,1]$, because $\left|3e^{8\pi i (t - \tfrac 1 2 )}\right| > \left| e^{12\pi i(t - \tfrac 1 2 )} + 1\right|$.
Thus, having shown that $f \circ \gamma$ and $g$ are homotopic within $\mathbb C^\star$, we deduce that they have the same winding number. As you say, the winding number of $g$ is obviously $2$, so this must be the winding number of $f \circ \gamma$ too.
Best Answer
The $(-20)$ term is dominant in the unit disk: With $f(z)= z^8 + 5z^7 -20$ and $g(z) =20$ you have for all $z$ on the boundary of the unit disk $$ |f(z) + g(z)| = |z^8 + 5z^7| \le |z|^8 + 5 |z|^7 = 6 < 20 = |g(z)| \le |f(z)| + |g(z)| $$ which means that $f$ and $g$ have the same number of zeros in the unit disk: none.
The same result can be achieved without Rouché's theorem: For $|z| \le 1$ is $$ |f(z)| = |z^8 + 5z^7 -20| \ge 20 - |z^8| - 5|z^7| \ge 20-1-5 = 14 > 0 \, . $$