On the subject of categorical versus set-theoretic foundations there
is too much complicated discussion about structure that misses the
essential point about whether "collections" are necessary.
It doesn't matter exactly what your personal list of mathematical
requirements may be -- rings, the category of them, fibrations,
2-categories or whatever -- developing the appropriate foundational
system for it is just a matter of "programming", once you understand
the general setting.
The crucial issue is whether you are taken in by the Great Set-Theoretic
Swindle that mathematics depends on collections (completed infinities).
(I am sorry that it is necessary to use strong language here in order to
flag the fact that I reject a widely held but mistaken opinion.)
Set theory as a purported foundation for mathematics does not and cannot
turn collections into objects. It just axiomatises some of the intuitions
about how we would like to handle collections, based on the relationship
called "inhabits" (eg "Paul inhabits London", "3 inhabits N"). This
binary relation, written $\epsilon$, is formalised using first order
predicate calculus, usually with just one sort, the universe of sets.
The familiar axioms of (whichever) set theory are formulae in first order
predicate calculus together with $\epsilon$.
(There are better and more modern ways of capturing the intuitions about
collections, based on the whole of the 20th century's experience of algebra
and other subjects, for example using pretoposes and arithmetic universes,
but they would be a technical distraction from the main foundational issue.)
Lawvere's "Elementary Theory of the Category of Sets" axiomatises some
of the intuitions about the category of sets, using the same methodology.
Now there are two sorts (the members of one are called "objects" or "sets"
and of the other "morphisms" or "functions"). The axioms of a category
or of an elementary topos are formulae in first order predicate calculus
together with domain, codomain, identity and composition.
Set theorists claim that this use of category theory for foundations
depends on prior use of set theory, on the grounds that you need to start
with "the collection of objects" and "the collection of morphisms".
Curiously, they think that their own approach is immune to the same
criticism.
I would like to make it clear that I do NOT share this view of Lawvere's.
Prior to 1870 completed infinities were considered to be nonsense.
When you learned arithmetic at primary school, you learned some rules that
said that, when you had certain symbols on the page in front of you,
such as "5+7", you could add certain other symbols, in this case "=12".
If you followed the rules correctly, the teacher gave you a gold star,
but if you broke them you were told off.
Maybe you learned another set of rules about how you could add lines and
circles to a geometrical figure ("Euclidean geometry"). Or another one
involving "integration by parts". And so on. NEVER was there a "completed
infinity".
Whilst the mainstream of pure mathematics allowed itself to be seduced
by completed infinities in set theory, symbolic logic continued and
continues to formulate systems of rules that permit certain additions
to be made to arrays of characters written on a page. There are many
different systems -- the point of my opening paragraph is that you can
design your own system to meet your own mathematical requirements --
but a certain degree of uniformity has been achieved in the way that they
are presented.
We need an inexhaustible supply of VARIABLES for which we may substitute.
There are FUNCTION SYMBOLS that form terms from variables and other terms.
There are BASE TYPES such as 0 and N, and CONSTRUCTORS for forming new
types, such as $\times$, $+$, $/$, $\to$, ....
There are TRUTH VALUES ($\bot$ and $\top$), RELATION SYMBOLS ($=$)
and CONNECTIVES and QUANTIFIERS for forming new predicates.
Each variable has a type, formation of terms and predicates must respect
certain typing rules, and each formation, equality or assertion of a
predicate is made in the CONTEXT of certain type-assignments and
assumptions.
There are RULES for asserting equations, predicates, etc.
We can, for example, formulate ZERMELO TYPE THEORY in this style. It has
type-constructors called powerset and {x:X|p(x)} and a relation-symbol
called $\epsilon$. Obviously I am not going to write out all of the details
here, but it is not difficult to make this agree with what ordinary
mathematicians call "set theory" and is adequate for most of their
requirements
Alternatively, one can formulate the theory of an elementary topos is this
style, or any other categorical structure that you require. Then a "ring"
is a type together with some morphisms for which certain equations are
provable.
If you want to talk about "the category of sets" or "the category of rings"
WITHIN your tpe theory then this can be done by adding types known as
"universes", terms that give names to objects in the internal category
of sets and a dependent type that provides a way of externalising
the internal sets.
So, although the methodology is the one that is practised by type theorists,
it can equally well be used for category theory and the traditional purposes
of pure mathematics. (In fact, it is better to formalise a type theory
such as my "Zermelo type theory" and then use a uniform construction to
turn it into a category such as a topos. This is easier because the
associativity of composition is awkward to handle in a recursive setting.
However, this is a technical footnote.)
A lot of these ideas are covered in my book "Practical Foundations of
Mathematics" (CUP 1999), http://www.PaulTaylor.EU/Practical-Foundations
Since writing the book I have written things in a more type-theoretic
than categorical style, but they are equivalent. My programme called
"Abstract Stone Duality", http://www.PaulTaylor.EU/ASD is an example of the
methodology above, but far more radical than the context of this question
in its rejection of set theory, ie I see toposes as being just as bad.
There’s one issue underlying a lot of the discrepancies between people’s answers, I think:
How are we defining “$f$ is a function $s \to V$”, where $s$ is a set and $V$ is a (possibly proper) class?
(hence also, how we define subsequent things like “a small-category-indexed diagram of sets”) There are at least two main options here:
$f$ is a class of pairs, such that…
$f$ is a set of pairs, such that…
At least in most traditional presentations, I think it’s defined as the latter, but some people here also seem to be using the former. The answer to this question depends on which we take.
If we take the “a function is just a class” definition, then as suggested in the original question, and as stated in François’ answer, we definitely have some big problems without replacement: Set is no longer complete and co-complete, etc. (nor are the various important categories we construct from it); we can’t easily form categories of presheaves; and so on. Under this approach, we certainly get crippling problems in the absence of replacement.
On the other hand, if we take the “a function must be a set” definition, we get some different problems (as pointed out in Carl Mummert’s comments), but it’s not so clear whether they’re big problems or not. We now can form limits of set-indexed families of sets; presheaf categories work how we’d hope; and so on. The problem now is that we can’t form all the set-indexed families we might expect: for instance, we if we’ve got some construction $F$ acting on a class (precisely: if $F$ is a function-class), we can’t generally form the set-indexed family $\langle F^n(X)\ |\ n \in \mathbb{N} \rangle$.
This is why we still can’t form something like $\bigcup_n \mathcal{P}^n(X)$, or $\aleph_\omega$. On the other hand, such examples don’t seem to come up (much, or at all?) outside set theory and logics themselves! Most mathematical constructions that do seem to be of this form — e.g. free monoids $F(X) = \sum_n X^n$, and so on — can in fact be done without replacement, one way or another.
Now… I seem to remember having been shown an example that was definitely “core maths” where replacement was needed; but I can’t now remember it! So if we take this approach, then we certainly still lose something; but now it’s less clear quite how much we really needed what we lost.
(This approach is very close to the question “What maths can be developed over an arbitrary elementary topos?”.)
Best Answer
In contrast to what some of the other answers seem to be saying, I believe that size issues play a very important role in category theory. Consider, for instance, the notion of complete category, i.e. a category having all small limits. Most "naturally-ocurring" categories, such as sets, groups, categories, etc. are complete (and cocomplete), and the ability to construct small limits and colimits is extremely important. However, these are all large categories, and a classic proof due to Freyd shows that in fact any small complete category must be a preorder (i.e. any two parallel arrows are equal). Thus, one of the most basic notions of category theory (completeness) becomes trivial if you aren't careful with size distinctions.
I also feel that more mathematicians should pay attention to set-theoretic issues, especially in category theory, and I wrote an unfortunately lengthy note myself on the subject, akin to Murfet's and Easwaran's pages linked to in Greg's answer.
However, for purposes of learning category theory, I don't think one should pay too much attention to any of this stuff. I think all you need to know, beyond naive set theory, is that some collections are "too big to be sets" (like the collection of all sets) but we can still manipulate them more or less as if they were sets, and we call them "classes." NBG and MK formalize this nicely with the "Limitation of Size" axiom: a class is a set if and only if it is not bijective with (i.e. "is not as big as") the class of all sets.