The underlying formalism, whatever it is, should be introduced very carefully.
The important thing is to teach concepts and methods. Formalism should be used whenever it is helpful for the student, but it should never be used on the grounds that "this is what mathematics really is" (it is not!) or "it is more precise this way" (but completely obfuscating!), or some such.
The examples you listed have very little to do with set theory. The $\epsilon\delta$ definitions are not hard because of set theory but because it is hard for humans to understand the difference between $\forall \exists$ and $\exists \forall$. The general idea of a map is not bound to set theory either, and neither are equivalence relations or quotients. All of these can be done in type theory, for example. In fact, if you open a random textbook it will read like type theory, not set theory.
If there is one things we do want to pick up from various formalisms, it is that we should not use broken notation. We should teach properly the difference between free and bound variables (something many mathematicians cannot get a handle on because they were taught 17th century syntax), that by itself will clean up a lot of confusion. We should always, always distinguish a function $f$ from its value $f(x)$ at $x$. We should never confuse an expressions $x^2 +1$ with a function $x \mapsto x^2 + 1$, or think that polynomials are functions. We should never say that one variable depends on another. And so on.
I taught freshmen logic and set theory. The first time around I naively explained what a formal proof was. They all learnt how to produce formal proofs, but had no idea what they were for. The next time around I taught logic and sets informally, and made the mistake of teaching logic first, and then sets "axiomatically". As it turns out "pure logic" is too pure, we had nothing to speak about. The third time around I "covered" logic in two lectures and went on to teaching "sets". I introduced things as we went along, and it was mostly about how to read and write proofs, how to transcribe a statement from natural language to a formula and back, how to deal with unions, intersections, subsets, quotients, direct and inverse images, etc. Mostly things which they are supposed to learn by osmosis in other courses. I don't think I got very far, though, and I am still not sure what the point of the course is, other than to hit students with very abstract stuff early on.
The discussion in the comments kind of went off the rails, but the point I meant to make by linking to dichotomy between nice objects and nice categories is that you can get lots of examples by starting with a nice category and restricting to a full subcategory by imposing some condition on the objects that isn't preserved by categorical operations. Fields are one example; manifolds are another. So are CW-complexes and Kan complexes, or more generally the category of cofibrant and/or fibrant objects in any model category.
There's a certain genericity to this class of examples, of course, since any (small) category can be embedded in its presheaf category, which is almost maximally nice.
Best Answer
I don't quite understand your question, but if you're asking whether category theorists should worry about set-theoretic problems the answer seems to be "sometimes". I'm not an expert in this area, but it seems that people tend to avoid universal constructions like limits over large diagrams, and in other cases, people assume the existence of strongly inaccessible cardinals. This seems to avoid standard contradictions, but I must confess that I've never checked such arguments.
I don't know many references for this question. Lurie discusses some constructions in section 5.4 of Higher topos theory.