The most immediately obvious relation to category theory is that we have a category consisting of types as objects and functions as arrows. We have identity functions and can compose functions with the usual axioms holding (with various caveats). That's just the starting point.
One place where it starts getting deeper is when you consider polymorphic functions. A polymorphic function is essentially a family of functions, parameterised by types. Or categorically, a family of arrows, parameterised by objects. This is similar to what a natural transformation is. By introducing some reasonable restrictions we find that a large class of polymorphic functions are in fact natural transformations and lots of category theory now applies. The standard examples to give here are the free theorems.
Category theory also meshes nicely with the notion of an 'interface' in programming. Category theory encourages us not to look at what an object is made of, but how it interacts with other objects, and itself. By separating an interface from an implementation a programmer doesn't need to know anything about the implementation. Similarly category theory encourages us to think about objects up to isomorphism - it doesn't precisely proclaim which sets our groups comprise, it just matters what the operations on our groups are. Category theory precisely captures this notion of interface.
There is also a beautiful relationship between pure typed lambda calculus and cartesian closed categories (CCC). Any expression in the lambda calculus can be interpreted as the composition of the standard functions that come with a CCC: like the projection onto the factors of a product, or the evaluation of a function. So lambda expressions can be interpreted as applying to any CCC. In other words, lambda calculus is an internal language for CCCs. This is explicated in Lambek and Scott. This means for instance that the theory of CCCs is deeply embedded in Haskell, because Haskell is essentially pure typed lambda calculus with a bunch of extensions.
Another example is the way structural recursion over recursive datatypes can be nicely described in terms of initial objects in categories of F-algebras. You can find some details here.
And one last example: dualising (in the categorical sense) definitions turns out to be very useful in the programming languages world. For example, in the previous paragraph I mentioned structural recursion. Dualising this gives the notions of F-coalgebras and guarded recursion and leads to a nice way to work with 'infinite' data types such as streams. Working with streams is tricky because how do you guard against inadvertently trying to walk the entire length of a stream causing an infinite loop? The appropriate dual of structural recursion leads to a powerful way to deal with streams that is guaranteed to be well behaved. Bart Jacobs, for example, has many nice papers in this area.
I believe this is a question that has not been adequately explored.
I view topos-like set theory and ZF-like set theory as exposing two
faces of the same subject. In ZF-like theory, sets come equipped with
a "membership" relation $\in$, while in topos-like theory, they do
not. The former, which I call "material set theory," is the standard
viewpoint of set theorists, but the second, which I call "structural
set theory," is much closer to the way sets are used by most
mathematicians.
However, the two viewpoints really contain exactly the same
information. Of course, any material set theory gives rise to a
category of sets, but conversely, as J Williams pointed out, from
the topos of sets one can reconstruct the class of well-founded
relations. With suitable "axioms of foundation" and/or
"transitive-containment" imposed on either side, these two
constructions set up an equivalence between "topoi of sets, up to
equivalence of categories" and "models of (material) set theory, up to
isomorphism."
Of course, it happens quite frequently in mathematics that we have two
different viewpoints on one underlying notion, and in such a case it
is often very useful to compare the meaning of particular statements
from both viewpoints. Usually both viewpoints have advantages and
disadvantages and each can easily solve problems that seem difficult
to the other. Thus, I see a tremendous and (mostly) untapped
potential here, if the ZF-theorists and topos theorists would talk to
each other more. How much of the structure studied by ZF-theorists
can be naturally seen in categorical language? Does this language
provide new insights? Does it suggest new structure that hasn't yet
been noticed?
One example is the construction of new models. Many of the
constructions used by set theorists, such as forcing, Boolean-valued
models, ultrapowers, etc. can be seen very naturally in a
topos-theoretic context, where category theory gives us many powerful
techniques. I personally never understood set-theoretic
forcing until I was told that it was just the construction of the
category of sheaves on a site. From this perspective the "generic"
objects in forcing models can be seen to actually have a universal
property, so that for instance one "freely adjoins" to a model of set
theory a particular sort of set (say, for instance, a set with
cardinality strictly between $\aleph_0$ and $2^{\aleph_0}$), with
exactly the same universal property as when one "freely adjoins" a
variable $x$ to a ring $R$ to produce the polynomial ring $R[x]$.
On the other hand, some constructions seem more natural in the world
of material set theory, such as Gödel's constructible universe.
I don't know what the category-theoretic interpretation of that is.
So both viewpoints are important.
Another example is the study of large cardinals. Many or most large
cardinal axioms have a natural expression in structural terms. For
example, there exists a measurable cardinal if and only if there
exists a nontrivial exact endofunctor of $Set$. And there exists a
proper class of measurable cardinals if and only if $Set^{op}$ does
not have a small dense subcategory. Some people at least would argue
that Vopenka's principle is much more naturally formulated in
category-theoretic terms. I have
asked
where there are nontrivial logical endofunctors of $Set$; this seems
to be a sort of large-cardinal axiom, but it's unclear how strong it
is. It seems possible to me that categorial thinking may suggest new
axioms of this sort and new relationships between old ones.
Best Answer
As a (slowly) recovering category-phobe, allow me to suggest that you change the way you think of category theory. Specifically, don't think of category theory as a "theory". A theory in mathematics generally consists of three components: a collection of related definitions, a collection of nontrivial theorems about the objects defined, and a collection of interesting examples to which the theorems apply. To learn a theory is to understand the proofs of the main theorems and how to apply them to the examples.
Category theory is different: there is an incredibly rich supply of definitions of examples, but very few theorems compared to other established "theories" like group theory or algebraic topology. Moreover, the proofs of the theorems are almost trivial (the Yoneda lemma is one of the most important theorems in category theory and it is not even called a "theorem"). A consequence of this is that you don't have to sit around reading a category theory book before you make contact with the language of categories: the very act of understanding how people express results from "ordinary" mathematics in the language of categories and functors is learning category theory.
So now I'll try to answer your question. It is possible to work in nearly any area of pure mathematics without much category theory, and most areas have people everywhere on the category theory spectrum (with the possible exception of algebraic geometry, wherein the language of derived functors is basically built into the foundations). Analysis in particular seems to be somewhat resistant (but not immune) to categorification, and if you are really committed to avoiding categories then you might consider exploring the more analytic aspects of what interests you (e.g. geometric PDE's or analytic number theory).
But before you make that commitment, try to find some examples of categorical language in action in what you already understand. The sheer ingenuity of it all might change your mind.