I'll take a swing at answering the question that I think you are trying to ask. I'll formulate it as follows:
If foundations are important as the name suggests and the so-called
"foundational crisis" suggests, why do so few mathematicians concern
themselves much with them nowadays. If foundations aren't important,
then why was there a "foundational crisis" and a significant effort to
create foundations?
tl;dr "Foundations" and ZFC were created to solve a fairly specific problem (founding real analysis), which they did. Now we don't worry about the problem, so many mathematicians don't have much reason to "faff about" with foundations.
The first thing to note is the obvious statement that mathematics has been done before, during, and after the establishment of ZFC as a foundational system. Just as clearly, very little mathematics prior to the establishment of ZFC has been deemed "incorrect" since its establishment. (Even the parts that arguably may have been have often been "revitalized" in modern treatments, sometimes utilizing other foundational approaches, e.g. "infinitesimals".)
So the first point is "doing math" doesn't require a foundational system as witnessed by the fact that math was being done for thousands of years before the advent of ZFC or anything like it. This is also witnessed by the fact that you can learn quite a bit of math today without concerning yourself much with the details of ZFC.
My understanding of the situation near the "foundational crisis", which may well be wrong - I'm no math historian - is there was a fairly specific group that wanted something like set theory: real analysts (as we'd call them nowadays). My reading of the situation is that it was the controversies and vagaries in real analysis that sparked mathematical (as opposed to philosophical) interest in foundations. Intuitions about "real numbers", "functions", "continuous functions" were not enough for the mathematicians of the day to converge on questions like what the Fourier transform of the constant function should be or whether it should even exist. This also raised the possibility that the notion of "real numbers" itself might be incoherent.
This led to the early work on defining the reals and defining a notion of function. (There were also philosophical motivations for this work that were likely partially independent, but I suspect that without the issues in real analysis mathematicians would have largely ignored such work.) Of course, from there Russell's paradox scuttled the still largely intuitive conception of naive set theory. This likely also scuttled the idea that we could rely on mathematical intuition alone and reinforced the possibility that, e.g., the real numbers really could be built on quicksand. They certainly were in naive set theories. Then we had 40 years of many proposed foundational systems, modifications to those systems, critiques of those systems, meta-logical analyses of those systems, and hands-on work using the systems. Presumably the majority of mathematicians of the time were at most spectators to this. They continued to plod on doing math the way they'd always done it likely without much concern if they were, say, a (non-analytic) number theorist.
I would say the main go/no-go issue for a set theory of that time would be whether it could found real analysis, i.e. construct the real numbers, construct a notion of continuous function, and prove widely accepted results like Heine-Borel. Jumping to the modern day, yes, it is the case that the mere existence of any acceptable foundation removes much of the urgency of the "foundational crisis". Most mathematicians during the "foundational crisis" didn't care about sets, they cared about real numbers and continuous functions. Given some other (non-set-theoretic) framework to assuage their concerns, they would have had little interest in set theory. Nowadays, students (perhaps unfortunately...) don't have these concerns in the first place, so they have little reason to devote much or any time to foundations. Set theory is usually taught in a naive way with some warnings. The language and tools from set theory are useful even without it being a foundational system, so it's not ignored entirely.
The variety of foundational systems, the fact most of them are also capable of serving as a foundation for real analysis and most other branches of math, and the fact that ZFC itself goes far beyond what is needed by most mathematicians means most of mathematics doesn't really depend on the specific details of the underlying foundational system. For example, while finding an inconsistency in ZFC would be big news, it's hard to imagine that it would also impact all other foundational systems that are capable of supporting e.g. real analysis. It is likely that most results would be unaffected and the ones that were affected would be relatively easily adapted and still "morally true". Maybe an extra assumption is added, say.
Another aspect of this is that there are many results in more solidly grounded fields like number theory that have proofs that use mathematical objects in less solidly grounded fields like real analysis. To the extent these results have "elementary" proofs within the more solidly grounded field, we have a web of justification for the validity of non-trivial aspects of the less solidly grounded field. This puts some limits on how "wrong" we could be in those fields before we'd have to be "wrong about everything".
It is true that, if $\mathsf{ZFC}$ is consistent, $\mathsf{CH}$ is undecidable within it. This is just one example of a more general fact: any consistent recursively enumerable first-order theory at least as strong as Peano arithmetic contains a statement undecidable in that theory. This is the first of Gödel's incompleteness theorems; the second gives an example, namely a statement calling the theory consistent.
The real question isn't whether $\mathsf{ZFC}$ has known limitations of this kind; of course it does. The question is which other statements we should add as axioms. The $\mathsf{C}$ in $\mathsf{ZFC}$ is the axiom of choice, which is itself undecidable in $\mathsf{ZF}$. The history of set theory has seen much broader support in favour of adding $\mathsf{AC}$ than in favour of subsequently adding $\mathsf{CH}$.
Why? Well, let's look at some of the differences:
- Although $\mathsf{AC}$ was initially much more controversial than it is today, it has come to enjoy broad support for the simple reason that, although it has some counter-intuitive consequences such as Zermelo's well-ordering theorem, its negation has "even worse" consequences such as trichotomy violation.
- Not only is $\beth_1=\aleph_1$ (i.e. the $\mathsf{CH}$) undecidable in $\mathsf{ZFC}$; so is $\beth_1=\aleph_n$ for any positive integer $n$. Why would we adopt the first as an axiom? By contrast, $\mathsf{AC}$ doesn't have an infinite family of obvious counterparts that feel equally feasible.
- One good thing about the $\mathsf{CH}$ is that it's a special case of a more general idea that looks nice, the $\mathsf{GCH}$ ($\beth_\alpha=\aleph_\alpha$ for all ordinals $\alpha$). There is some interest in adding $\mathsf{GCH}$ to $\mathsf{ZFC}$, but just $\mathsf{CH}$ on its own? That's an unpopular compromise between $\mathsf{ZFC}$ (which is weak enough for some people's tastes) and $\mathsf{ZFC+GCH}$ (which is strong enough for some other people's tastes).
- Lastly, outside of set theory $\aleph_1$ usually doesn't even come up as a concept, and so $\mathsf{CH}$ has no obvious benefit when founding any other areas of mathematics. (There are exceptions, e.g. nonstandard analysis uses $\mathsf{CH}$ to consider real infinitesimals.) By contrast, $\mathsf{AC}$ has uses all over the place, e.g. proving every vector space has a basis.
Best Answer
The question is broad, so I'll just try to address one of the sub-questions. I'll change it a little bit to allow for the possibility of pluralism:
As an example, I'll try to explain why set theory is a foundation of mathematics, and what it means for it to be one. This isn't intended to exclude other possible foundations, although they tend to be subsumed by set theory in a sense discussed below.
Based on one of the OP's comments above, I'll treat the mention of computability in the question figuratively rather than literally. Roughly speaking, a Turing-complete system of computation is one that can simulate any Turing machine, and so by the Church–Turing thesis, can simulate any other (realistic) system of computation. Therefore a Turing-complete system of computation can be taken as a "foundation" for computation.
As far as this question is concerned, I think that under the appropriate analogy between computation and mathematical logic, the notion of simulation of one computation by another corresponds to the notion of interpretation of one theory in another. By an interpretation of a theory $T_1$ in a theory $T_2$ I mean an effective translation procedure which, given a statement $\varphi_1$ in the language of $T_1$, produces a corresponding statement $\varphi_2$ in the language of $T_2$ such that $\varphi_1$ is a theorem of $T_1$ if and only if $\varphi_2$ is a theorem of $T_2$. So a mathematician working in the theory $T_1$ is essentially (modulo this translation procedure) working in some "part" of the theory $T_2$.
In these terms, a foundation of mathematics could be reasonably described as a (consistent) mathematical theory that can interpret every other (consistent) mathematical theory. However, it follows from the incompleteness theorems that no such "universal" theory can exist. But remarkably, the set theory $\mathsf{ZFC}$ can interpret almost all of "ordinary" (non-set-theoretic) mathematics. For example, if I'm working in analysis and I prove some theorem about continuous functions, then my theorem and its proof can in principle be translated into a theorem of $\mathsf{ZFC}$ and its proof in $\mathsf{ZFC}$. (Under this translation, a continuous function is a set of ordered pairs, which are themselves sets of sets, satisfying a certain set-theoretic property, etc.)
This means that set theory can serve as a foundation for most of mathematics. For set theory itself, no one theory (e.g. $\mathsf{ZFC}$) can serve as a universal foundation. But we can informally define a hierarchy of set theories ($\mathsf{ZFC}$, $\mathsf{ZFC} + {}$"there is an inaccessible cardinal", $\mathsf{ZFC} + {}$"there is a measurable cardinal", etc.) called the large cardinal hierarchy, strictly increasing in interpretability strength, which seems to be practically universal in the sense that every "natural" mathematical theory, set-theoretic or otherwise, can be interpreted in one of these theories. (The reason this doesn't violate the incompleteness theorem is that the large cardinal hierarchy is defined informally in an open-ended way, so we can't just take its union and get a universal theory.)
Your last point about the continuum hypothesis raises an important question: what if the various candidate set theories branch off in many different directions rather than lining up in a neat hierarchy? Well, it turns out so far that the natural theories we consider do line up in the interpretability hierarchy, even if they do not line up in the stronger sense of inclusion. For example, the methods of inner models and of forcing used to prove the independence of $\mathsf{CH}$ from $\mathsf{ZFC}$ also give interpretations of the two candidate extensions $\mathsf{ZFC} + \mathsf{CH}$ and $\mathsf{ZFC} + \neg \mathsf{CH}$ by one other (and by $\mathsf{ZFC}$ itself,) showing that they inhabit the same place in the interpretability hierarchy. If you believe $\mathsf{CH}$ and I believe $\neg\mathsf{CH}$ we can still interpret each others results, rather than dismissing them as meaningless.