As someone working in the field of mathematical physics, I believe that for a person with your interests it is somewhat better to be a graduate student in the math department. The reason is while there are quite a few mathematicians which regard physics as an important source of problems and insights, there are far fewer physicists who think that mathematical physics is worth working on. In particular, foundational issues in QFT and strings are not very popular with most physicists.
My opinion is that physicists transferred from study of "individual objects" to that of "large systems" where the order arises from limit probability laws rather than from simple deterministic formulae and from the study of something "readily observable" to something that is, essentially, "a purely mathematical object" invisible to a direct experiment. This brought them to the realm traditionally reserved for pure mathematicians. And, of course, with their eagerness to use whatever tools they have available in any way that is short of total lunacy, they went on to make predictions, many of which could be confirmed experimentally, leaving a long trail of successes and failures in their wake for mathematicians to explain.
I do not know the situation with the string theory and low dimensional topology but I have some idea about what's going on in random matrices (thanks to Mark Rudelson and his brilliant series of lectures) and in percolation/random zeroes (thanks to Stas Smirnov and Misha Sodin). The thing that saves physicists from making crude mistakes there is various "universality laws".
Here is a typical physicist's argument (Bogomolny and Schmidt). You want to study the nodal domains of a random Gaussian wave $F$ (the Fourier transform of the white noise on the unit sphere times the surface measure). Let's say, we are in dimension 2 and want just to know the typical number of nodal lines (components of the set $\{F=0\}$) per unit area. The stationary random function $F$ has only a power decay of correlations. However,
we ignore that and model it with a square lattice that has the same length per unit area as $F$ (this is a computable quantity if you use some standard integral geometry tricks). Now, at each intersection of lattice lines, we choose one of the two natural ways to separate them (think of the intersection as of a saddle point with the crossing lines being the level lines at the saddle level). Then, we get a question (still unresolved on the mathematical level, by the way) about a pure percolation type model. Thinking by analogy once more, we get a numerical prediction.
From the viewpoint of a mathematician, this all is patented gibberish. There is no way to reduce one process to another (or, at least, no one has the slightest idea how this could be done as of the moment of this writing). Still, the Nature is kind enough to make the answers the same or about the same for all such processes and Mathematics is powerful enough to provide an answer (or a part of an answer) for some models, so the physicists run a simulation, and, voila, everything is as they predicted and we are left with 20 years or so worth of work to figure out what is really going on there.
I'm not complaining here, quite the opposite: this story is really quite exciting and the work mentioned is both real and fascinating. We are essentially back to the days when Newton tried to explain the nature of gravity looking at Kepler's laws trying various options and separating what works from what doesn't. I'm only saying that the famous "physicists' intuition", which is so overrated, is actually just the benevolence of Nature. Why should the Nature be so benevolent to us remains a mystery and I know neither a physicist, nor a mathematician, who could shed any light on that. The best explanation so far is contained in Einstein's words "God is subtle, but not malicious", or, in a slightly less enigmatic form, "Nature conceals her mystery by means of her essential grandeur, not by her cunning".
Best Answer
Perhaps it would not be out of place to quote Miles Reid's Bourbaki seminar on the McKay correspondence here:
"The physicists want to do path integrals, that is, they want to integrate some "Action Man functional" over the space of all paths or loops $ \gamma : [0; 1] \rightarrow Y $. This impossibly large integral is one of the major schisms between math and fizz. The physicists learn a number of computations in finite terms that approximate their path integrals, and when sufficiently skilled and imaginative, can use these to derive marvellous consequences; whereas the mathematicians give up on making sense of the space of paths, and not infrequently derive satisfaction or a misplaced sense of superiority from pointing out that the physicists' calculations can equally well be used (or abused!) to prove 0 = 1. Maybe it's time some of us also evolved some skill and imagination. The motivic integration treated in the next section builds a miniature model of the physicists' path integral,..."