Gödel's view on CH changed over his lifetime, so it is hard to give a comprehensive answer to the question about his reasoning. It evolved over the years, and toward the end of his life he even came to believe that the CH may be true (although he still believed the GCH was false).
Fortunately, there is a three-volume collected works of Gödel, and most of what I say here is gleaned from the commentary in there, as well as some secondary sources I gave in the comments below the questions.
First off, I should say that while many of Gödel's philosophical ideas on set theory from the mid 40s onward (i.e. after his development of the $L$ hierarchy and proof of the consistency of AC and GCH) are regarded as important, even if they weren't all super influential at the time, his ideas on the specific question of the absolute truth of CH are mostly considered dead ends.
With that said, the natural place to start is his proof of the consistency of GCH in the late 30s. He did this by defining the constructible sets $L,$ and showing that they form a model of ZFC + GCH. In his initial development, Gödel believed that the great clarification of the set concept given by his axiom of constructibility was perhaps the missing piece needed to decide our set theoretical questions. This, of course, would amount to a belief that CH is true.
However he quickly reversed this position and came to what has since been the dominant view among Platonists that the axiom of constructibility is obviously false. In his 1947 expository paper What is Cantor's Continuum Problem?, he likens the constructible sets to a model of non-Euclidean geometry constructed within Euclidean geometry: while this establishes the consistency of non-Euclidean geometry, it has no bearing on the "true" Euclidean universe. The axiom does clarify the notion of a set, but it does so by placing severe restrictions on what a set is, saying they all need to be obtained from transfinite iteration of simple constructive operations. This, to Gödel and the majority of set theorists after him, seemed to be the exact opposite of what a principle guiding the concept of an arbitrary set should do.
In the same passage, Gödel argued that the CH was probably not provable from ZFC. Essentially, although it may be the the wrong clarification, the axiom of constructibility does seem to be a very strong clarification of what sets there are, and it would be odd if a question like CH did not require this clarification (or one of similar magnitude) in its solution. (Of course on this point, Gödel was resoundingly correct.)
Now to finally touch on the issue in your question. As a secondary argument that CH is not provable, he asserts that it is probably false. His argument is fairly thin: he states without much elaboration that he finds several descriptive set theory consequences of CH to be implausible. (For instance the existence of uncountable absolute measure zero sets and Sierpinski sets.) My descriptive set theory is pretty weak, so I don't know quite what to make of this, but eminent set theorist Donald Martin has said
While Gödel's intuitions should never be taken lightly, it is very hard to see
that the situation is different from that of Peano curves, and it is even hard for some of us to see why the examples Gödel's cites are implausible at all.
(Peano curves are a counterintuitive construction that does not require CH that Gödel claims without much substantiation that the situation is different for.)
So although much of this article was insightful (including a lot of stuff I didn't touch on about the direction forward in finding new axioms), Gödel's arguments for the falsity of CH were not taken up by the mathematical community.
Gödel didn't have much output between 1947 and the advent of forcing in the early 60s (although he had attempted with some progress to establish the consistency of the negation of choice). Cohen's proof was more than just a confirmation that ZFC could not prove CH: it showed that $2^{\aleph_0}$ could consistently take arbitrarily large values, and that the meager facts that we already knew about the size of the continuum were essentially all that ZFC could tell us. This intensified what was already a suspicion in the set theory community that the continuum was probably very large.
While he was rightfully in awe of Cohen's work, Gödel had of course long believed that CH was not provable and had been looking in other directions. He had expressed hope that large cardinal axioms would decide the CH, but shortly after the advent of forcing it was discovered by Levy and Solovay that this would not work. (Despite this, the large cardinal program has been very fruitful in general, and did strongly refute the axiom of constructibility.) Meanwhile, he had hit upon an old idea of Hausdorff that seemed to him to produce tractable conjectures that were intuitively true and informative on the continuum.
This work is the subjuct of an unpublished handwritten note in 1970s in which Gödel claims to have a convincing argument that $2^{\aleph_0}=\aleph_2.$ Details can be found in the collected works or Kanamori's Gödel and Set Theory. Interestingly, once he had formulated his axioms (known as "rectangle axioms") it was discovered that, amongst other issues, they actually implied the CH rather than $2^{\aleph_0}=\aleph_2.$ Undeterred, he came around to the belief that the CH was probably true after all, and that in any event this approach would give strong evidence that the continuum was small (no larger than $\aleph_2$).
Although this was considered a failure (an interesting one by some), Kanamori notes that it rhymes with the broader history of set theory post-Cohen. After years of believing the continuum was large, set theorists began to seriously consider some deep principles that would imply (of all things) $2^{\aleph_0}=\aleph_2.$ And sure enough, shortly after Kanamori wrote this, another principle came into vogue that would imply CH. Which goes to show that thinking about the truth of CH itself has largely given way to thinking about what deeper and more general principles should hold, and then accepting whatever they imply about the CH.
Best Answer
First of all, the Completeness Axiom is considerably more powerful than the statement "the square root of $2$ exists". The Completeness Axiom promises the existence of every irrational number. Before accepting the Axiom, it would be perfectly reasonable for a mathematician to accept that $\sqrt{2}$ exists but deny the existence of, say, $\pi$.
Second, "recognized the importance of" and "invented" are very different things. What Bolzano did was notice that everyone was using this axiom, but that no one was actually putting it into words. Long before Bolzano, people accepted the existence of irrationals; they just didn't have an axiom to point to and say "this is why".
If we want to go back far enough that irrational numbers actually weren't accepted, we have to go all the way back to ancient Greece - as soon as you have the Pythagorean Theorem, you have to accept the existence of $\sqrt{2}$ (unless you're okay with claiming that isosceles right triangles don't exist!). Before that, mathematicians certainly did believe that there was no such number - I can't find a resource specifically claiming "there is no square root of two", but the ancient Greeks did work under the assumption "every number is the ratio of two whole numbers".