As alluded-to by Qiaochu Y. above, and as I can personally attest, PDE arise in the modern theory of automorphic forms. Superficially/historically, this might be viewed as a formal generalization of "holomorphic" to "eigenfunction for Laplace-Beltrami operator". Indeed, already c. 1947, Maass showed that real quadratic fields' grossencharacter L-functions arose as Mellin transforms of "waveforms", Laplace-Beltrami eigenfunctions on $\Gamma\backslash H$, a complementary result to his advisor Hecke's result that $L$-functions for complex quadratic extensions of $\mathbb Q$ arose from holomorphic modular forms.
The spectral theory of automorphic forms, from Avakumovic, Roelcke, and Selberg c. 1956, in effect decomposes $L^2(\Gamma\backslash H)$ with respect to the invariant Laplacian, descended from the Casimir operator on the group $SL_2(\mathbb R)$, which (anticipating theorems of Harish-Chandra) almost exactly corresponds to decomposition into irreducible unitary representations.
The Selberg trace formula, and Langlands' and Arthur's, as well as Jacquet's "relative" trace formula, do afford an interpretation as spectral decompositions of various integral operators, rather than differential operators. Nevertheless, or "however", some aspects of the situation that are clumsy, because of their "extreme" features, but interesting for applications for the same reason, from that viewpoint are amenable to thinking about solutions of (invariant) inhomogeneous PDEs with distributional "targets". A typical scenario is a "Helmholtz" equation (a wave equation Fourier-transformed in the time parameter), $(\Delta-\lambda)u=f$. Among other cases of interest, the case that $f$ is an (automorphic) delta is very useful in various number-theoretic applications, such as proving "subconvex" bounds: Anton Good sketched this application already in 1983 (and Diaconu and I treated $GL_2$ over number fields recently... implicitly using this idea, although reference to classical special functions gave a shorter argument for the official version).
Philosophizing a bit, such experiences, and continuing ones of a related sort, indicate to me that geometrically meaningful, that is, group-invariant, "PDE" are a natural/obvious extension of "calculus"... so that, in particular, their natural solutions in Sobolev spaces (etc) are "natural objects", whether or not they are classical special functions, or entirely elementary.
(One can't help but note that there is an understandable, if unfortunate, human tendency to declare and understand "turf", so that one chooses one's own, and stays away from others'. Similarly, "experts" on subject X do not favor outsiders' appropriating bits of it "for applications", as though anything other than a life-long dedication could penetrate the mysteries... One may read about medieval European "guilds" and their protection of their "secrets".)
As a methodological philosophizing: my own experience tells me that means of description are useful. That is, structural, meaningful characterization of objects is good. Saying that something is a solution of a natural (group-invariant?...) PDE is a strong, meaningful constraint. Ergo, helpful/good.
The small rant at the end: the usual style of seemingly-turf-respecting narrowness is not so good for genuine progress, nor even for individual understanding.
Best Answer
I find Tim Gowers' "two cultures" distinction to be relevant here. PDE does not have a general theory, but it does have a general set of principles and methods (e.g. continuity arguments, energy arguments, variational principles, etc.).
Sergiu Klainerman's "PDE as a unified subject" discusses this topic fairly exhaustively.
Any given system of PDE tends to have a combination of ingredients interacting with each other, such as dispersion, dissipation, ellipticity, nonlinearity, transport, surface tension, incompressibility, etc. Each one of these phenomena has a very different character. Often the main goal in analysing such a PDE is to see which of the phenomena "dominates", as this tends to determine the qualitative behaviour (e.g. blowup versus regularity, stability versus instability, integrability versus chaos, etc.) But the sheer number of ways one could combine all these different phenomena together seems to preclude any simple theory to describe it all. This is in contrast with the more rigid structures one sees in the more algebraic sides of mathematics, where there is so much symmetry in place that the range of possible behaviour is much more limited. (The theory of completely integrable systems is perhaps the one place where something analogous occurs in PDE, but the completely integrable systems are a very, very small subset of the set of all possible PDE.)
p.s. The remark Qiaochu was referring to was Remark 16 of this blog post.