The definite answer to your question is:
There is no mathematicaly precise, commonly accepted definition of the term "regularization procedure" in perturbative quantum field theory.
Instead, there are various regularization schemes with their advantages and disatvantages.
Maybe you'll find Chapter B5: Divergences and renormalization of my theoretical physics FAQ at http://arnold-neumaier.at/physfaq/physics-faq.html illuminating. There I try to abstract the common features and explain in general terms what is needed to make renormalization work. The general belief is that the details of the regularization scheme don't matter, though in fact it is known that sometimes some regularization schemes give apparently incorrect results.
This is to be expected since the unregularized theory is ill-defined, and can be made well-defined in different ways, just as a divergent infinite series can be given infinitely many different meanings depending how you group the terms to sum them up.
If at any time in the future there will be a positive answer to your question, it will be most likely only when someone found a logically sound nonperturbative definition of the class of renormalizable quantum field theories.
On the other hand, if you want to have a mathematically rigorous treatment of some particular regularization schemes for some particular theories, you should read the books by (i) Salmhofer, Renormalization: an introduction, Springer 1999,
and (ii) Scharf, Finite quantum electrodynamics: the causal approach, Springer 1995.
I will use the expressions "virtual particles" and "internal lines in a Feynman diagram" interchangably in this answer.
This interpretation fails because you can draw Feynman diagrams in both position and momentum space. When you draw them in momentum space and squint really hard, you might be able to convince yourself they have something to do with "Fourier components of the field", but the Fourier transform simply appears because you obtained the momentum space expressions from the position space expressions by a Fourier transform - it's not a characteristic of virtual particles.
In particular, virtual particles do not appear because we expand the field. They appear because we expand the interaction part of the time evolution operator - the n-point function we want to compute is something like $\langle \mathcal{T}\prod_{i = 1}^n \phi_i(x_i)\exp(-\mathrm{i}\lambda\int V(\phi(x))\mathrm{d}^d x)\rangle$ and the virtual particles appear when we expand the exponential as its power series and represent the resulting expressions graphically.
That we expand the time evolution operator suggests another "intuition" however: Virtual particle represent (fictitious) intermediate states over which we must sum. The time evolution operator there essentially means that when we compute the n-point function in an interacting theory, we must not only compute the "naive" n-point function, but also that in the presence of 1, 2, 3, 4, (i.e. finitely many) interactions (the vertices). At weak coupling $\lambda$, each interaction term suppresses the contribution of the summand it appear in, meaning the diagrams with few vertices contribute the most.
Heuristically: If you compare this to usual time-dependent perturbation theory in ordinary quantum mechanics (semi-randomly picked reference from a Google search [pdf]), it should be familiar: Expand the time evolution operator into its Dyson series, insert identies $1 = \sum_m \lvert m \rangle\langle m \rvert$, compute up to desired/feasible order. If we interpret $V(\phi)$ as the vertices, then the internal lines correspond to the inserted identities, meaning you may view virtual particles simply as generic "intermediate states" over which we must sum. You may even draw the same types of diagrams to organize such perturbation series in general.
What I think the blog post by Matt Strassler you're citing is getting at is that there is, regardless of all "virtual particles", of course an actual intermediate state during a QFT scattering, however complicated to describe it may be. He's saying that "virtual particles" are what physicists call that intermediate state which is...close enough, but more precisely we need to keep in mind that the intermediate states of perturbation theory aren't the same as the actual state of the system, they are computationally convenient fictions.
Finally, let me reiterate what I've said many times: Statements like "A static EM field is made out of virtual particles" are nonsense, since virtual particles are a tool in perturbation theory, not a fundamental entity in the theory. If you can manage to compute the relevant quantities without perturbation theory, then you'll never use the notion of virtual particle. Since physics does not depend on the specific mathematical method chosen to evaluate the relevant quantities, such statements have no basis in general QFT, and fail completely at strong coupling where no perturbative expansion is accessible. The only rigorous sense in which a static EM field is "made out of virtual particles" is that the Coulomb potential may be recovered from a diagram with a virtual particle in it. (Note that also in that case the Fourier transform is because we're computing stuff in momentum space, not because of the virtual particle as such.)
Best Answer
Like you said, we can include gravity perturbatively in the framework of low-energy effective QFT, as reviewed in reference 1. This works because gravity is extremely weak at the energies that characterize modern particle-physics experiments. But the interest in quantum gravity revolves around nonperturbative/high-energy/strong-field issues, like the holographic principle and the informaion-loss paradox, both of which were already known in the 1970s (references 2,3,4) and were surely on Distler's mind in 1982.
Thanks to universality, very different theories can become indistinguishable from each other at sufficiently low resolution. Low-energy experiments can only fix the first several terms in the lagrangian on which perturbation theory is based. That's what allows us to include gravity in the Standard Model in the sense of low-energy effective theory (reference 1), and I'm guessing this was also the basis for Georgi's assertion. Terms of higher order in the cutoff are not resolved, so we cannot attack the interesting questions about quantum gravity — which are nonperturbative/high-energy/strong-field — by extrapolating upward from the low-energy effective theory.
Even if it was fair at the time, Georgi's "waste of time" judgement is obsolete now, because now we have approaches to studying quantum gravity that don't rely on extrapolating upward from a low-energy effective theory. Perturbative string theory is tightly constrained by numerous anomaly cancellation requirements, which are nonperturbative. Fully nonperturbative formulations like AdS/CFT are also available. (See references 5 and 6 for perspectives about the situation in the more realistic case of asymptotically de Sitter spacetime, which is not understood as well yet.) In hindsight, Georgi/Distler's statement
seems to be true in an even stronger sense in string theory. Here's an excerpt from section 2.2 in reference 7:
Whether this "stringy" phenomenon is our enemy or our friend, it at least corroborates the idea that the interesting questions about quantum gravity are not things we can study properly by decoupling it from everything else.
Donoghue (1995), Introduction to the Effective Field Theory Description of Gravity (https://arxiv.org/abs/gr-qc/9512024)
Bekenstein (1973), Black holes and entropy, Physical Review D 7, 2333-2346
Hawking (1975), Particle creation by black holes (https://projecteuclid.org/euclid.cmp/1103899181)
Hawking (1976), Breakdown of predictability in gravitational collapse, Phys. Rev. D 14, 2460–2473
Witten (2001), Quantum Gravity In De Sitter Space (https://arxiv.org/abs/hep-th/0106109)
Banks (2010), Supersymmetry Breaking and the Cosmological Constant (https://arxiv.org/abs/1402.0828)
Palti (2019), The Swampland: Introduction and Review (https://arxiv.org/abs/1903.06239)