See Gödel's Incompleteness Theorems.
"Preliminary" formulation :
First incompleteness theorem : Any consistent formal system $F$ within which a "certain amount of elementary arithmetic" can be carried out is incomplete; i.e., there are statements of the language of $F$ which can neither be proved nor disproved in $F$.
Precise formulation :
Gödel's First Incompleteness Theorem : Assume $F$ is a formalized system which contains Robinson arithmetic $\mathsf Q$. Then a sentence $G_F$ of the language of $F$ can be mechanically constructed from $F$ such that:
If $F$ is consistent, then $F ⊬ G_F$.
If $F$ is $1$-consistent, then $F ⊬ ¬G_F$.
$\mathsf {ZFC}$ is precisely such a formalized system $F$ (i.e. "a formal system within which a 'certain amount of elementary arithmetic' can be carried out").
Gödel's original proof in his Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I ("On Formally Undecidable Propositions of Principia Mathematica and Related Systems I") does not use $\mathsf {ZFC}$ nor $\mathsf {PA}$.
His proof is relative to a "variant" of Principia Mathematica system; but his proof - in addition to esatblish the result now know as Gödel's Incompleteness Theorems - provides also a method applicable to many formal systems, provided that they satisfy some "initial conditions".
All the systems known as $\mathsf Q, \mathsf {PA}, \mathsf {ZFC}$ satisfy these conditions.
Relevant to your question, you can see :
It can be interesting to note that Melvin Fitting's thesis advisor was Raymond Smullyan.
Perhaps "surprising" is no longer the right word; what is surprising to one person may not be surprising to another. The fundamental theorem of calculus is another result which is sometimes claimed to be "surprising" but which may not surprise everyone. So I don't want to dwell on the "surprising" part, but I want to address a few other questions from the post.
One point is that, even if the incompleteness theorem is not surprising, its proof is far from obvious - particularly the theorem as stated in the question, which requires Rosser's trick in addition to Gödel's argument. If the incompleteness theorem was as obvious as the fact that a group can be commutative or noncommutative, we might expect the proof to be simpler.
The second point is that, although it is true that (e.g.) the axioms of a group are not complete, they are easy to make complete. For example, the theory of torsion free, divisible Abelian groups is complete, consistent, and axiomatizable, as is the theory of algebraically closed fields of a fixed characteristic. The theory of real closed fields is axiomatizable, consistent, and complete, as is Tarski's axiomatization of Euclidean geometry.
So there is something different about arithmetic, compared to real closed fields or geometry. In that sense, the incompleteness theorems may not be surprising, but they are fundamental. They show that we cannot hope to attain the kind of axiomatization for arithmetic that we can obtain for the field of real numbers or for geometry.
Many of the theories in the mathematical literature describe a class of structures (all groups, all fields, all posets, etc.) and so we don't expect them to be complete. But other theories might be intended to describe a particular structure - the real line, the Euclidean plane, the natural numbers. In these cases, once we pick the signature, there is already a complete theory at hand, namely the set of all true sentences of the structure in that signature. So the question is not so much about whether a particular theory is complete (as in the penultimate paragraph of the post), but whether we could in principle write a complete ''and effective'' set of axioms for the theory of the structure and signature we are looking at. For the specific structure $\mathbb{N}$, the incompleteness theorem shows that this is impossible, but goes beyond that to show more.
Best Answer
In simple terms the way I understand it is this. If you pick any consistent set/system $A$ of axioms (which obeys certain conditions), and thus build a (math) theory, there will always be statements $S$ in this theory which you can formulate and which are true, but you cannot prove just by using your set $A$ of axioms.
This is Goedel's 1st incompleteness theorem.
That's why it's called incompleteness theorem. Because any consistent system of axioms is not complete i.e. cannot prove all the statements which can be formulated.
There's also a 2nd incompleteness theorem by Goedel which states that no set/system of axioms can prove its own consistency.
Note that both formulations given here are rough and loose.
See also:
How Goedel's incompleteness theorems work?
Book:
Ernest Nagel, James Newman, "Gödel's Proof", 1958