Caveat: in order to give you an overview, I've been vague/sloppy in several places.
Well the basic link to representation theory is that modular forms (and automorphic forms) can be viewed as functions in representation spaces of reductive groups. What I mean is the following: take for example a modular form, i.e. a function $f$ on the upper-half plane satisfying certain conditions. Since the upper-half plane is a quotient of $G=\mathrm{GL}(2,\mathbf{R})$, you can pull $f$ back to a function on $G$ (technically you massage it a bit, but this is the main idea) which will be invariant under a discrete subgroup $\Gamma$. Functions that look like this are called automorphic forms on $G$. The space all automorphic forms on $G$ is a representation of $G$ (via the right regular represenation, i.e. $(gf)(x)=f(xg)$). Basically, any irreducible subrepresentation of the space of automorphic forms is what is called an automorphic representation of $G$. So, modular forms can be viewed as certain vectors in certain (generally infinite-dimensional) representations of $G$. In this context, one can define the Hecke algebra of $G$ as the complex-valued $C^\infty$ functions on $G$ with compact support viewed as a ring under convolution. This is a substitute for the group ring that occurs in the representation theory of finite groups, i.e. the (possibly infinite-dimensional) group representations of $G$ should correspond to the (possibly infinite-dimensional) algebra representations of its Hecke algebra. This type of stuff is the basic connection of modular forms to representation theory and it goes back at least to Gelfand–Graev–Piatestkii-Shapiro's Representation theory and automorphic functions. You can replace $G$ with a general reductive group.
To get to more advanced stuff, you need to start viewing modular forms not just as functions on $\mathrm{GL}(2,\mathbf{R})$ but rather on $\mathrm{GL}(2,\mathbf{A})$, where $\mathbf{A}$ are the adeles of $\mathbf{Q}$. This is a "restricted direct product" of $\mathrm{GL}(2,\mathbf{R})$ and $\mathrm{GL}(2,\mathbf{Q}_p)$ for all primes $p$. Again you can define a Hecke algebra. It will break up into a "restricted tensor product" of the local Hecke algebras as $H=\otimes_v^\prime H_v$ where $v$ runs over all primes $p$ and $\infty$ ($\infty$ is the infinite prime and corresponds to $\mathbf{R}$). For a prime $p$, $H_p$ is the space of locally constant compact support complex-valued functions on the double-coset space $K\backslash\mathrm{GL}(2,\mathbf{Q}_p)/K$ where $K$ is the maximal compact subgroup $\mathrm{GL}(2,\mathbf{Z}_p)$. If you take something like the characteristic function of the double coset $KA_pK$ where $A_p$ is the matrix with $p$ and $1$ down the diagonal, and look at how to acts on a modular form you'll see that this is the Hecke operator $T_p$.
Then there's the connection with number theory. This is mostly encompassed under the phrase "Langlands program" and is a significantly more complicated beast than the above stuff. At least part of this started with Langlands classification of the admissible representation of real reductive groups. He noticed that he could phrase the parametrization of the admissible representations say of $\mathrm{GL}(n,\mathbf{R})$ in a way that made sense for $\mathrm{GL}(n,\mathbf{Q}_p)$. This sets up a (conjectural, though known now for $\mathrm{GL}(n)$) correspondence between admissible representations of $\mathrm{GL}(n,\mathbf{Q}_p)$ and certain $n$-dimensional representations of a group that's related to the absolute Galois group of $\mathbf{Q}_p$ (the Weil–Deligne group). This is called the Local Langlands Correspondence. The Global Langlands Correspondence is that a similar kind of relation holds between automorphic representations of $\mathrm{GL}(n,\mathbf{A})$ and $n$-dimensional representations of some group related to Galois group (the conjectural Langlands group). These correspondences should be nice in that things that happen on one side should correspond to things happening on the other. This fits into another part of the Langlands program which is the functoriality conjectures (really the correspondences are special cases). Basically, if you have two reductive groups $G$ and $H$ and a certain type of map from one to the other, then you should be able to transfer automorphic representations from one to the other. From this view point, the algebraic geometry side of the picture enters simply as the source for proving instances of the Langlands conjectures. Pretty much the only way to take an automorphic representation and prove that it has an associated Galois representation is to construct a geometric object whose cohomology has both an action of the Hecke algebra and the Galois group and decompose it into pieces and pick out the one you want.
As for suggestions on what to read, I found Gelbart's book Automorphic forms on adele groups pretty readable. This will get you through some of what I've written in the first two paragraphs for the group $\mathrm{GL}(2)$. The most comprehensive reference is the Corvallis proceedings available freely at ams.org. To get into the Langlands program there's the book an introduction to the Langlands program (google books) you could look at. It's really a vast subject and I didn't learn from any one or few sources. But hopefully what I've written has helped you out a bit. I think I need to go to bed now. G'night.
Concerning advanced differential geometry textbooks in general:
There's a kind of a contradiction between "advanced" and "textbook". By definition, a textbook is what you read to reach an advanced level. A really advanced DG book is typically a monograph because advanced books are at the research level, which is very specialized. Anyway, these are my suggestions for DG books which are on the boundary between "textbook" and "advanced". (These are in chronological order of first editions.)
- Bishop/Crittenden, "Geometry of manifolds" (1964). Quite advanced, although not too difficult, despite the 1964 date.
- Cheeger/Ebin, "Comparison theorems in Riemannian geometry" (1975). This is on the boundary between textbook and monograph. Definitely advanced, despite the 1975 date.
- Greene/Wu, "Function theory on manifolds which possess a pole" (1979). Monograph/textbook about function theory on Cartan-Hadamard manifolds, including extensive coverage of Kähler manifolds.
- Schoen/Yau, "Lectures on Differential Geometry" (1994). This is about as advanced as it gets. You need to read at least 5 other DG books before starting this one.
- Theodore Frankel, "The geometry of physics: An introduction" (1997, 1999, 2001, 2011). This has lots of advanced DG, but in the physics applications, not so much on topological DG questions.
- Peter Petersen, "Riemannian geometry" (1998, 2006). Very definitely advanced. You need to read at least 3 other DG books before this one.
- Serge Lang, "Fundamentals of differential geometry" (1999). This is definitely advanced, although it nominally starts at the beginning. It's what I call a "higher viewpoint" on DG. Very thorough and demanding.
- Morgan/Tián, "Ricci flow and the Poincaré conjecture" (2007). Advanced monograph on the Poincaré conjecture solution, but written almost like a textbook.
- Shlomo Sternberg, "Curvature in mathematics and physics" (2012). Definitely advanced. On the boundary between DG and physics.
I would say that all of these books are beyond the John M. Lee and Do Carmo textbook level.
Best Answer
I would suggest Procesi's Lie Groups, as a text that introduces algebraic groups with minimal prerequisites. Chapter 7 "Algebraic Groups" is
This is followed by Chapter 8 "Representation Theory",