I am just going to give an example of what I mean using Skolem's Paradox. I don't want to get into Skolem;s Paradox itself or its "resolution."
Skolem's showed that in first-order formulations of ZFC, whether some set A is countable depends on what is in the model. For example, take a model M of ZFC (assuming there is one). Let M satisfy the statement "S | S is countable." This means there exists a bijection from S to {naturals} in M (i.e. there is a particular set of ordered pairs in M). Now, remove all and only those bijections from M and call this new model M'. Assume M' is still a model of ZFC. Is S still countable? No. Countability is relative to the elements of M's domain.
How how does a mathematical object, e.g. a countable set, relate to the real world? Is it in the real world? Is is instantiated in the real world? What is meant when mathematicians say (e.g. as Cantor's Theorem says) that uncountable sets exist?
That is, what is the relation of math-objects to the real world?
Best Answer
Once mathematics began dealing properly with infinite objects it was no longer about the reality, but rather about abstract ideas.
Our "natural" intuitions (i.e. those we have from a pre-mathematical education time) are often very wrong about the infinite, to list some examples:
The list itself is infinite. It gets even larger if you wish to consider it in early 1900's eyes where the axiom of choice were still researched thoroughly.
However mathematics no longer deals solely with describing the real world, it deals with deductions from assumptions. Once accepting that it seems that a lot of the problems with infinities dissipate, as they follow from definition.
There comes a new problem with foundations of mathematics, the independence of claims, in particular the set theoretical ones. How can a set be countable in one model and uncountable in another? Let me use, once again, my usual analogies from field theory.
Suppose $F$ is a field (of characteristics $0$ if you prefer). What is the size of $\{x\in F\mid\exists n\in\mathbb N^+: x^n=1\}$
In the rational numbers the answer is $2$, in the rational numbers adjoined by a complex unit root of order $3$ the answer is $4$; in the algebraic closure of the rationals the answer is countably infinite. In the complex numbers you don't increase the size of this set, but you find a lot more transcendental elements on the unit circle which you can't even describe so nicely.
Note that field theory cannot express in a single formula the notion of being a unit root; but it can express the notion of being a unit root of order, say, $72$ or less. This should give us enough examples ($\mathbb Q$ still has only two; different extensions have four, five, etc.) of a specific definable set which changes in size between the models.
Why does no one complain when they are told that "in this field there are more unit roots than in that field"? My guess is that we are being educated to accept that "all numbers live in $\mathbb C$", so some are rationals, some are algebraic, etc. and thus different fields would have different amount of unit roots.
But set theory deals with sets, is this a surprise that different models of set theory would have different sets and if we pass from one model to a smaller model we may lose some of the information? No. If you study some axiomatic set theory you find out it's not surprising at all. It's what you'd expect, much like the way you may lose some unit roots in passing to a smaller field.
Now you are probably thinking, "he must be cheating me somewhere, because I feel completely fine with the unit root example, but it's impossible for sets to be countable here and uncountable there!". Well, sticking to first-order logic, you have to ask yourself what is the language that you use to describe the axioms and the model. In field theory you essentially describe the operations and the polynomials which have a solution in the field. In set theory you only have $\in$, but you describe a more complicated creature.
Is it a surprise that we have computers and an amoeba have only one cell? No, we are a far more complicated creature. Set theory is far more complicated, as a theory, than field theory. It should not be a surprising understanding that some of the things it can say about objects in the universe are more complicated. Since those are complicated it often seems that there should be some "canonical answer", but so far there is none. Whether it is good or bad, I can't tell. I hope there won't be a canonical answer because I enjoy the plethora of models, much like (I suppose) people studying measure theory enjoy the plethora of measures and spaces attached to those.
I will finish with one last point, Skolem tried to show in his paradox not that there is an inherent problem with set theory describing the world but rather that there is an inherent problem with using first-order logic to describe set theory. As it happens to be, he actually made clear the distinction between "internal" and "external" points of view in logic.