It's complete if there is only one basis of common eigenvectors. That means, There is only one basis in which the matrices are diagonal matrices.
Let's start with only 2: operators $A$ and $B$.
If $[A,B]=0$, there is at least one orthonormal basis of common eigenvectors.
If the eigenvalues of $A$ have no degenerancy, then the basis is unique (except for global phase factors), and hence the set is complete.
If $A$ has degenerate eigenvalues, then they form subspaces (the matrix has boxes along the diagonal). $B$ acts on each subspace without merging with others.
Inside every subspace, you can find a basis of $B$ which makes sub-sub-spaces (sub-boxes).
If those subspaces are more than 1-dimensional, then the system is not complete, but there's a third commuting observable $C$ that might make the matrices diagonal.
There might be more than 1 CSCO with different eigenvalues.
For a given CSCO, eigenvalues of all operators specify one only common eigenvector.
As for the practical question, you can show it in the particular case.
But one can show that any spherically-symmetric setup satisfies $[H,\vec{L}]=0$, and therefore there's $[H, L^2]=0, \ [H, L_z]=0$.
This is because any system which is invariant under rotations verifies that $H$ conmutes with rotations, and rotations are a function of $\vec{L}$, so $[H, \vec{L}]=0$. It's long to prove, but it is a very beautiful topic.
Note: invariant under rotations refers that you'll get the same result by
a) Let it evolve in time and then rotate it.
b) Rotate it first and then let it evolve.
Best Answer
They don't have to, $S_z$ and the identity commute. The former has two eigenvalues. The latter has one. Other examples exist too, such as the operator, $H=p^2/2m+e/r$ which has an infinite number of eigenvalues but can commute with operators with a finite number of eigenvalues such as both operators listed above. However, if you look at the dimension of each eigenspace and add up all those dimensions for each eigenvalue then you get the same number for any hermitian operator (regardless of whether they commute) because since they are hermitian that sum just gives you dimension of the whole space. And that sum is just the dimension of the whole space and it happened regardless of whether they commuted, so it is really besides the point. The answer is that there is no real relationship in general for the number of eigenvalues of operators that commute.
They don't have to, $S_z$ and the identity commute. If the operators act on a spin space the former has two eigenvectors (modulo scalars) and the latter has uncountably many (a whole two dimensional eigenspace of eigenvectors with uncountably many directions). Again you could count the dimension of each eigenspace and add them up, but you'll just get the dimension of the whole space again. There will still literally be two directions that are eigen to one operator and uncountably many directions that are eigen to the other operator. And adding the dimensionality of the eigenspaces for each operator will again just give you the dimension of whole space. And again it happenes regardless of whether they commuted, so it is really besides the point. The answer is that there is no real relationship in general for the number of eigenvector directions of operators that commute.
Nothing, the sets could be finite or countably infinite or for eigenvectors there can be uncountably many. Even $AB$ could have uncountably many different directions of eigenvectors. If you have not just two but have a collection of mutually commuting observables where you can't add more observables that also commute with those (a CSCO) then the vectors that are common eigenvectors to all the operators will have fixed directions so now there is a fixed set of directions. And the cardinality of those directions will be the dimension of the whole space but now it is actually related to the number of eigenvector directions that are eigen to all the operators, the eigenvectors that are common to all the operators give you real directions. And you can count them, but counting them will just give you the dimensionality of the whole space.