I'm learning this stuff myself so take this with a large grain of salt but a commenter on this question suggested Warner, Foundations of Differentiable Manifolds and Lie Groups.
At a glance it looks like it goes through some of the usual topics but then does the de Rahm theorem using sheaves, so you might get along with it. Apart from Ch.5, though, I'm not sure how different it is from a standard treatment. It's a GTM book with minimal prereqs, and if you already know about sheaves it's probably a fairly gentle read.
I'd be interested to know how those in the know regard this text in relation to (what I take to be) the more usual textbooks.
The short answer is: it depends! To do differential geometry you don't really need category theory at all, and the same could (nearly) be said for some flavors of algebraic geometry. That said, some people (myself included) learn things best from a categorical standpoint. If you get excited whenever people mention universal properties, and are happiest defining things in terms of a functor that they represent, then starting with some category theory may be a good thing for you. In that case, I would recommend working through the first chapters of the classic Categories for the working mathematician. In particular, you want a solid understanding of limits, adjoint functors, and the relation between the two.
Now, if you don't even know group theory yet, starting with category theory is a bad idea. It would be best to start with some abstract algebra, using one of the standard texts.
It may be that you are a more normal mathematician for whom "categories first" or "algebra first" is not a good idea. In this case, if you are interested in differential geometry, the best thing to do would be to learn differential geometry, and only spend time on other topics as necessary.
Algebraic geometry can be almost entirely non-categorical, or hyper-categorical depending on what you are interested in doing. How much category theory you will need to know depends primarily on your own tastes in algebraic geometry.
Best Answer
Indeed, (smooth) manifolds form a category, with the morphisms being the (smooth) maps between manifolds.
If $M$ is a (smooth) manifold and $C(M)$ is the space of (real or complex) (smooth) functions on $M$, then $C$ is a contravariant functor from $M$ to $C(M)$. Notice that $C(M)$ naturally sits in several categories: sets, commutative groups, vector spaces, algebras, rings - you choose which one is of interest to you by dropping some structure off $C(M)$.
(Notice that $C$ takes the morphism $f : M \to N$ into the morphism $C(f) : C(N) \to C(M)$ given by $C(f) (\phi) = \phi \circ f$, i.e. $C(f) = f^*$, the pull-back.)
A very well known category-flavoured book on differential geometry is "Natural Operations In Differential Geometry" by Ivan Kolár, Peter W. Michor, Jan Slovák (freely available online) - but don't expect the same level of category theory as in algebraic topology or algebraic geometry.
Another, more recent book, that presents differential geometry using an approach heavily inspired by algebraic geometry is "Manifolds, Sheaves and Cohomolgy" by Torsten Wedhorn.