Carrol's "Spacetime and Geometry: an Introduction to General Relativity" is a great complement to the mentioned book by Wald in the comments. While some proofs are relegated toward the appendix, it's great to get an overall picture. Plus it's more modern and hence mentions some recent results in cosmology, in particular on the subject of gravitational waves. I would say, turn to Wald for the more rigorous exposition but stick to Carrol as it's much easier to read.
For something at a more leisurely pace, I highly recommend Hartle's book "Gravity" which provides a fantastic amount of down to earth explanations while still being fairly rigorous. I would especially recommend it for its section on explaining how the moon affects the tides.
Since you seem unfamiliar with the nature of special relativity as the geometry of index 1 symmetric bilinear forms, aka "Lorentz forms", I think the best I can do is to give you a little bit of intuition about how these forms are connected to the reality of physical space time. This might allow you to realize the direction that one must go in order to formalize special relativity.
Beyond that, the best I can suggest is to suggest that you read on. I started with Resnick too, when I was a freshman, oh so many years ago, and it took me a long time until I properly appreciated the linear algebra basis of special relativity.
The starting point for special relativity in 2-dimensional space time is a 2-dimensional vector space $V$ equipped with a symmetric bilinear form $\langle v,w \rangle$ defined for all $v,w \in V$ such that the signature of this form is $(1,1)$. By definition, this means that $V$ has a basis in which the matrix of the form is $\begin{pmatrix}1 & 0 \\ 0 & -1\end{pmatrix}$.
Using these coordinates, we can identify $V$ with $\mathbb R^2$, and we can write the two coordinate functions as $x$, $t$. The first coordinate $x$ is usually called something like the "space-like" coordinate, and $t$ is called the "time-like coordinate".
Given $p,q \in V$ representing two space-time events, it is useful to ponder the meaning of the quantity
$$|q-p|^2 = -\langle q-p, q-p \rangle
$$
This quantity can be zero, positive, or negative, and I'll discuss the physical meaning of these possibilities.
Understanding a space time interval $|q-p|^2=0$ is a good start. Mathematically, this just means that the straight line through $p$ and $q$ has slope $\pm 1$. What $|q-p|^2=0$ means physically is that in order for a particle to move under inertia between the events $p$ and $q$, that particle must essentially be a photon: it must move at the speed of light (the coordinates have been normalized so that the speed of light equals $1$).
In general, the world line of a photon, or any particle moving in a straight line at the speed of light, is a path of the form
$$f(s) = p_0 + s \cdot v
$$
where the vector $v$ satisfies $\langle v,v \rangle = 0$. The appearance of this world line in the space-time $V$ is that it is a line of slope $\pm 1$.
The physical meaning of $|q-p|^2 > 0$ is that it is possible to move between the events $p,q$ by "real", inertial motion. Furthermore, if you move in that manner, and if your clock is ticking along, then the amount of time that your clock ticks off between $p$ and $q$ is equal to
$$|q-p| = \sqrt{|q-p|^2}
$$
Inertial motion from $p$ to $q$ is modelled by the parameterized path
$$f(s) = p + s u
$$
where $u = \frac{q-p}{|q-p|}$ (in order to be moving "forward" in time, the $t$ coordinate of the vector $u$ should be positive; in other words, $p$ is in the "past" of $q$ and $q$ is in the "future" of $p$). For example, if an object starts at $p_0=(x_0,t_0)$ and does not "move" in this coordinate system, then $u = (0,1)$ and the world line is
$$f(s) = (x_0,t_0+s)
$$
The specific example of an unmoving particle sitting at the origin $x_0=0$ is $p(s)=(0,s)$.
The physical meaning assigned to $|q-p|^2 < 0$ is that communication between the space time events $q$ and $p$ is impossible, because moving between them would require faster than light motion. So, for example, it is impossible to see what is happening at this exact moment one inch in front of your face.
Best Answer
Personally, I come from the opposite path as you: I began as a theoretical physicist and jumped recently into graduate mathematics. General relativity is the best place for a geometer to begin exploring physical applications of abstract theories like (pseudo-) Riemannian geometry, so do not get discouraged if you encounter "apparently ugly" or uninteresting things like "electromagnetism". Physics is not so much like mathematics in the sense that in the latter you can more or less remain isolated within a very narrow field without having to care for what is happening elsewhere; theoretical physics looks for the mathematical structures of Nature, and she is one interrelated reality. In fact, it was a correct understanding of electrodynamics within classical mechanics what led Einstein to develop special relativity, and an attempt to formulate gravity in a field theory manner like electromagnetism what led him to general relativity, so I really think you should by no means avoid electrodynamics, at least in its concepts and elementary field equations (which can be cast nicely into a pair of differential 2-form equations!). On the other hand, the typical first example of "the rest of physics" to be coupled to gravity is electromagnetism, either as an actor (e.g. deflected light propagating in curved spacetime) or as a source (i.e. density of electromagnetic energy being itself the source of the spacetime curvature).
If you have done a standard course in differential/Riemannian geometry you are more than prepared to enter general relativity in all its glory. It is very important coming from a mathematical background to choose carefully what titles and references consult to avoid wasting your time and efforts trying to make sense of the many prosaic informal treatments geared toward most non-rigorous physicists. Here you have a good reference list in order of difficulty/sophistication:
Lambourne, R.J.A. - Relativity, Gravitation and Cosmology.
This is an elementary introductory textbook, ideal for self-study and with a friendly outlook which makes it fun to read (good physics and mathematics need not be so dry all the time). It is a nice first read to any of the topics of the next title.
Carroll, Sean M. - Spacetime and Geometry: An Introduction to General Relativity.
This book is an excellent first introduction, and grew up from a set of notes freely available.
Misner, C.; Thorne, K.S.; Wheeler, J.A. - GRAVITATION.
The title says it all, it has been the standard bible of general relativity since the 1970s, a thousand pages of physics, digressions and mathematical geometric background explained excellently.
Straumann, N. - General Relativity.
It has been reedited with corrections, additions and retyped layout. This is one of my favourite books because it develops (now in long appendices) the geomeric and mathematical requisites in detail and proceeds with the theory thoroughly with much rigor, giving lots of applications in astrophysics. Perfect complement to Wald's. Besides the traditional Riemann curvature approach, it explains in detail the exterior (Cartan's) formulation, more suited for comparison to other gauge theories (and for quantum gravity) and for people trained in covariant connections on principal bundles.
Plebanski, J.; Krasinski, A.- Introduction to General Relativity and Cosmology.
A special new book, but perfectly suited for anyone wanting to learn general relativity aiming for quantum gravity (above all canonical, nonperturvative approaches). It develops some essentials which are not treated in older textbooks. Quite rigorous and formal but with lots of applications in cosmology (hence, great complement to Straumann's).
Wald, Robert M. - General Relativity.
This is the standard graduate mathematical introduction, including advanced topics like causality structure.
Hawking, S.; Ellis, G. - The Large Scale Structure of Space-Time.
The reference on mathematical relativity where most of the research of Hawking and Penrose was first proved.
De Felice, F.; Clarke. C. - Relativity on Curved Manifolds.
This is a very nice book from a purely advanced mathematical point of view, dealing with the most technical aspects of the formalism of space-time.
After all that, there are many other very specialized titles, like de Felice/Bini on classical measurements in space-time (very important to actually make sense of what can be "said" and "measured" in general relativity), Choquet-Bruhat on the mathematics of solutions to Einstein's equations, Griffiths/Podolsky on exact spacetime solutions... etc. Despite what many physicists (and even mathematicias) would say, I would, in my humble opinion, recommend you to avoid famous old references like Weinberg's "Gravitation and Cosmology", they develop general relativity from a purely non-geometric way in the style of a quantum field theorist, i.e. treating gravity as a spin 2 field, thus missing most of the point (diffeomorphism invariance!) and the modern mathematical machinery you have learned in differential geometry. That approach is very useful indeed for perturbation approximations, post-Newtonian consequences and attempts towards quantization, I would nevertheless argue that that approach maybe a conceptual dead end, despite its computational usefulness, since it explicitly parts with the original Einstein's understanding of the theory (see below about Rovelli's book and his emphasis on grasping the deep meaning of general relativity). All the nice applications proved in Weinberg's are now included in books like Straumann and Plebanski but with a modern approach and mathematical flavor, so you do not lose anything. And if you still want to learn also the field-theory version, I would recommend instead introductory books to superstrings and supergravity (see vol. 2 of the pair below) and above all the classic:
If you want to deepen your knowledge of differential geometry (not only for general relativity but for any other gauge theory), a very nice new and cheap introduction is:
Two new volumes have just been released with a focus on the geometric development of gravity, geared towards introducing supergravity and beyond. If that is your interest you should check them out.
Giuseppe Frè, P. - Gravity, A Geometrical Course. Vol. 1: Development of the Theory and Basic Physical Appications.
Giuseppe Frè, P. - Gravity, A Geometrical Course. Vol. 2: Black Holes, Cosmology and Introduction to Supergravity.
Last but not least, the following book is a masterpiece of exposition and clarification. Although its intention is introducing loop quantum gravity in the second half of the book, the first half is a conceptually deep, mathematically rigorous, treatment of the structure of classical and quantum mechanics, and general relativity. I can not emphasize enough HOW MUCH I have learned from the discussions on the meaning of mechanics, relationalism, the nature of space and time, general relativity and its background independence... By all means I encourage anybody to study in detail those sections of the book and open their minds to how physics was thought in the golden age (i.e. paying more attention to philosophical insights, conceptual meaning and empirical adequacy).
In particular, one becomes really surprised when one discovers that nowhere else, in any other physics textbook, is Einstein's "hole argument" mentioned and the meaning of spacetime coordinates (despite what some would say, the hole argument cannot be done in field theory approaches to gravity and their perturbative treatments! that is why they miss much of the conceptual content: the underlying differential manifold has no physical meaning, only correlations of observables define what measurable space and time really are). Straumann's book above is the only one with a brief mention to a rigorous definition of invariance and covariance; just this makes it a superior book!. General covariance and the meaning of spacetime coordinates are the cornerstone of general relativity, and their understanding is what kept Einstein confused and stuck from 1912 to 1915 in the search of the final field equations. Most physics books focus on mere "shut up and calculate" physics and not on the conceptual meaning that natural philosophy of science always accompanied in the past. This book will teach you that and much more. Concretely it will teach you the meaning of background independence, which in mathematical terms is the diffeomorphism invariance of the theory. General relativity has such a symmetry and that makes it more special than the other fields (that is why I recommended to avoid certain approaches and books which explicitly break its symmetry by perturbative approximations), making physics the study of relational interdependent degrees of freedom; in words of Einstein himself (from his original article in 1916):
... the requirement of general covariance takes away from space and time the last remnant of physical objectivity...
(The attempts to get a quantum gravity theory probably fail because of this, due to trying to deal with gravity as it were another force living in space-time... the whole particle physics community sometimes seems to have forgotten the conceptual lesson to be learnt from general relativity and quantum mechanics: they are both relational theories! all that can be found in Rovelli's book, and in his articles available at the arXiv; all this and more is the reason why I decided to switch from theoretical physics to mathematics; I apologize for bothering you or anybody else with these personal commentaries and recommendations, but they are relevant if one wants to understand what the problem might be after 60 years of unsuccessfully trying to quantize gravity).
I hope all of this is of any use to you. Good luck!