I was reading Goro Shimura's The Map of My Life.
He wrote the following quote in the book.
It made me come up with the title question.
In particular, is there any application of the theory of modular forms in physics?
A well known math-physicist Eugene Wigner was in our department,
and so I occasionally talked with him. He was pompous
and took himself very seriously. That is the impression shared by
all those who talked with him. At a departmental party, he asked
me what kind of mathematics I was doing. He asked that question
as if he met me for the first time. At that time I had been a full
professor and his colleague at least for six years, perhaps more. I
told him vaguely, “Well, mainly things related to modular forms.”
Then he said, “Oh, modular forms; we physicists don’t need such”
in a very contemptuous tone.
I can add that there are some physicists who are interested
in modular forms. Once Edward Witten attended my graduate
course on Siegel modular forms, and often asked sensible questions
in class.
Best Answer
To explain one aspect of the context of that interaction: Wigner was a very senior faculty member, had won a Nobel Prize, and was in his 60s. Shimura was a very young full professor.
Another aspect: Wigner's 1939 paper on the representation theory of (in effect) $SO(2,1)$, written to address issues of quantum mechanics, was the first substantive result on representation theory of non-abelian, non-compact groups. The 1947 paper by V. Bargmann, a physicist, was the second. No further progress on this until the 1950s when Harish-Chandra, a student of the physicist Dirac, began his systematic study of repn theory of semi-simple and reductive (Lie, and eventually, p-adic) groups.
Meanwhile, Shimura had almost single-handedly resurrected the arithmetic and algebraic-geometry aspects of holomorphic modular forms on higher rank groups, although Klingen and Hel Braun (a student of Siegel) had been ("quietly"?) working on the complex-analytic aspects, and Klingen's c. 1960 discussion of special values of (abelian) L-functions over number fields was very arithmetical. Perhaps Shimura's most special "early" contribution was to the possibility of expressing Hasse-Weil zeta functions of "Shimura curves" (as they are now known, a class generalizing "modular curves") as Mellin transforms of automorphic forms of some sort, etc.
Even the plausibility, of the Taniyama-Shimura conjecture would not be acknowledged by Weil until the mid-1960's, after his work on converse theorems. People then, and until the Wiles-Taylor work and others' in the mid-to-late 1990s, I think thought RH would be proven before Taniyama-Shimura. No one had any idea about RH, but they had even fewer [sic] ideas about Taniyama-Shimura.
Wigner would not have known about Weil's conjectures, nor the nascent algebraic geometry required to put them in any context. Shimura might not have believed that Harish-Chandra's repn theory, beginning with Wigner's result, would, as explained by Gelfand and his school, and Selberg, and taken up by Langlands et al, provide an over-arching context for not-necessarily-holomorphic automorphic forms, if not their "arithmetic".
The other "human" aspects of the situation we can imagine easily...
But, even beyond the human-foible aspect, it is absolutely not surprising that Shimura was not in awe of Wigner, and that Wigner had no reason to care much about Shimura's work.
Witten's interest was quite a few years later, after Shimura's, Selberg's, Harish-Chandra's, Langlands', and many others' work had made clear that the special objects studied in "number theory" strongly resembled the special objects of parts of physics. Not to mention that Witten is more of a "visionary" than many of us. And won a Fields Medal, so maybe he's a good mathematician, too? :)
From my personal viewpoint, apart from those historical observations, I note that the specific mathematics on arXiv that seems relevant to my concerns, second after "number theory", is the "math-ph" section.
As an example, the van Hove (et al) differential equations that (I hear...) model something about graviton interactions, are precisely the same genre of differential equations "in automorphic forms" that appear in various spectral-theoretic scenarios, going back to Anton Good's papers in the early 1980's, and continuing in various peoples' work today. Steven Miller at Rutgers, a guy who "does" automorphic forms, has actively collaborated with that physics groups, for example.
Indeed, Rudnick, Ueberschar, Marklof, and their collaborators often say that they are doing "mathematical physics", and are in "physics institutes", ... but their work looks to me like a study of number-theoretic aspects of harmonic analysis... which would extend to be "automorphic forms", if taken on to more difficult cases.
And, finally, probably autobiographies do not reliably involve scholarly reconsideration of much of anything at all, as they are reminiscences... so scientific accuracy is by far not guaranteed.