The tricky part is to explain the plateaus in $\sigma_{xy}$ and why they coincide with vanishings of $\sigma_{xx}$.
To make this more evident let us suppose first you had a system without disorder, say you have an interacting Galilean that is fully translationally invariant. Ideally this would be realized e.g. in a Torus. Then you can show, following Kohn's theorem, that the Hall conductivity is always exactly $\sigma_{xy}=\nu e^2/h$, and that $\sigma_{xx}=0$. This is completely independent of the microscopic state of the system, e.g. it woud be true if you had a Laughlin state or a Charge density wave or anything. It follows from the fact that the center of mass of the system as a whole moves as a free charged particle in electric and magnetic fields. Thus there are no plateaus in this translationally invariant limit: the Hall conductivity traces a straight line when you plot it vs $\nu$.
Therefore, what is non-trivial is to explain why the Hall conductivity gets "stuck" at $\sigma_{xy}=\nu_0 e^2/h$ when you change the filling $\nu$ slightly away from a special filling $\nu_0$, namely, the existence of a "plateau" and its coincidence with the vanishing of $\sigma_{xx}$. Essentially, the picture is that as you change the electron density (filling) away from $\nu_0$, they are added into states localized by the disorder and don't contribute to transport (A caveat is that there can be bulk states deep below the chemical potential that are delocalized and can contribute to current flow, see previous reponse).
In the Laughlin argument, which essentially applies only to non-interacting disordered electrons (i.e. only to the integer quantum Hall effect), after threading a flux quantum the Hamiltonian is the same, but the state has changed into an excited state. In this excited state, the occupation numbers of the single particle eigenstates have changed (we are imagining here the exact eigenstates of the single body problem in the presence of disorder). One needs to argue that the change of electron occupations occurs only at the edge and not in the bulk. This is why the argument relies not only on gauge invariance, but also in the existence of a bulk mobility gap. As long as the chemical potential lies in the bulk mobility gap, the electron occupation will only change at the edge after the flux insertion, and due to electron number conservation, this change is an exact integer which determines the Hall conductivity.
In the infinite system limit, each Landau level is infinitely degenerate since the degeneracy scales with area.
In reality, no system is infinite. Tong is simply taking a finite system of size $L_x\times L_y$ as to quantize the momenta. This is the whole system under consideration, not only a small cut-out. All states are counted.
The result is that the degeneracy scales with $L_xL_y$. You may now choose the system size to be anything you want.
Best Answer
The 0th Landau level also has a degeneracy of four as do other LLs. The significance of 0th LL is that it is shared by conduction and valence band with equal weight. That is to say: two of 0th LLs are electron like and two of them are hole-like (because the degeneracy is four, I imagine there are four "seperated" non-degenerate LLs at the same energy for pedagogical simplicity).
When an electron-like Landau-level is fully occupied by electrons, it contributes a unit conductance; when a hole-like Landau-level is fully occupied by holes, it contributes a negative unit conductance. (Assuming that the landau level has no degeneracy)
When an electron-like Landau-level is empty, it contributes no conductance (of course!); when a hole-like Landau-level is filled with electrons, it contributes no conductance.
Now let's see how the anomalous quantum hall effect arises:
When the 0th LL is fully occupied, the two electron-like LLs are filled with electron and two hole-like LLs are also filled with electron. Thus the conductance of the system is 2 unit conductance.
When the 0th LL is empty, the two electron-like LLs are empty and two hole-like LLs are filled with holes. Thus contribute to the conductance of -2 unit conductance.
Similarly, when both 0th and 1th LLs are also occupied with electrons, the total conductance is 4+2.
The origin of the anomalous Hall conductance
The anomalous Hall conductance arises because of the particle hole symmetry and the linear dispersion of graphene. For the 0th LL, because its energy is just 0, so it must be shared by the valence and conduction band.
However, for the parabolic dispersion, even if the original band have particle-hole symmetry. When forming LLs, the energy of lowest LLs formed by conduction band is 1/2, While the energy of lowest LL formed by valence band is -1/2. They do not mix.