There are really several questions here: (a) What is the renormalization group? Specifically the law of composition, etc. (b) How does the equation the OP gave relate to this?
Short Answer
It's a semigroup (see references below). The equation you wrote,
$$\tag{1}
\left[\mu \frac{\partial}{\partial \mu} + \beta \frac{\partial}{\partial g} + m \gamma_{m^2} \frac{\partial}{\partial m} - n \gamma_d \right] \Gamma^{(n)}(\{p_i\};g,m,\mu) = 0$$
is not "the renormalization group equation". It's the Callan-Symanzik equation, which is derived from the renormalization conditions (I cannot draw Feynman diagrams, so I cannot show you the rules per se).
For a derivation of the Callan-Symanzik equation from the renormalization group, see:
- Kleinert's Path Integrals in Quantum Mechanics, Statistics, Polymer Physics, and Financial Markets, specifically chapter 10 "Renormalization Group"; eprint
- John Cardy's lecture notes on QFT chapter 6
- Jan Louis' lecture notes on QFT, section 9
- Peskin and Schroeder's Introduction to Quantum Field Theory, ch. 12.
Handwavy Derivation of Renormalization group
We will consider the $\phi^4$ model. We want a "momentum cutoff" $\Lambda$. So we basically consider the Fourier transformed field $\phi(k)$ with nonzero components for $|k|<\Lambda$. We write
$$\tag{2a}Z_{\Lambda}
=\int\exp\left(-\int\left[\frac{1}{2}(\partial_{\mu}\phi)^{2}+\frac{1}{2}m^{2}\phi^{2}+\frac{\lambda}{4!}\phi^4\right]\,\mathrm{d}^{n}x\right)[D\phi]_{\lambda}$$
where
$$\tag{2b} [D\phi]_{\Lambda} = \prod_{|k|<\Lambda}\mathrm{d}\phi(k).$$
We now factor $Z$ into two components, the "high frequency" and "low frequency" parts. We want to integrate out the "high frequency" parts, so $Z$ only has an integral over "low frequency" parts. We take $0<b\leq1$.
Now the "high-frequency components" of $\phi(k)$ correspond to those with $k$ satisfying $b\Lambda\leq|k|\leq\Lambda$. We will transform $Z$ to depend only on frequencies $|k|\leq b\Lambda$. This is our transformation (or "law of composition", if you will). Since $0\lt b\leq1$, this transformation has no inverse...but it has an identity transformation when $b=1$. Hence to answer your question
Is the "renormalization group" a group?
The answer is "no". Now, lets see how to carry out this transformation (albeit slightly handwavy, just to show the milestones alont the way).
Label these components that we will integrate out as
$$\tag{3}\hat{\phi}(k) = \begin{cases}\phi(k) & \mbox{for $b\Lambda\leq|k|\leq\Lambda$}\\
0 & \mbox{otherwise}\end{cases}$$
Lets write $\tilde{\phi}(k)=\phi(k)-\hat{\phi}(k)$. Then observe the partition function becomes
$$\tag{4}
Z=\int D\tilde{\phi}\int D\hat{\phi}\exp\left(-\int\left[\frac{1}{2}(\partial_{\mu}\tilde{\phi}+\partial_{\mu}\hat{\phi})^{2}+\frac{1}{2}m^{2}(\tilde{\phi}+\hat{\phi})^{2}+\frac{\lambda}{4!}(\tilde{\phi}+\hat{\phi})^{4}\right]\,\mathrm{d}^{n}x\right)$$
The argument is that terms involving $\tilde{\phi}\hat{\phi}$ don't matter because components of different Fourier modes are orthogonal. So we integrate $\hat{\phi}$ over $b\Lambda\leq|k|\leq\Lambda$ and our partition function changes from
$$\tag{5a}
Z=\int D\tilde{\phi}\exp(-S[\tilde{\phi}])\int D\hat{\phi}\exp(-S[\phi])$$
arguing
$$\tag{5b}\exp(-\int\delta\mathcal{L}_{eff}(\phi)\,\mathrm{d}^{n}x) = \int D\hat{\phi}\exp(-S[\phi])$$
we get
$$\tag{5c}
Z=\int D\tilde{\phi}\exp(-\int[\mathcal{L}(\tilde{\phi})+\delta\mathcal{L}_{eff}(\phi)]\,\mathrm{d}^{n}x)$$
This transformation is parametrized by $b$, and cannot be undone. But we could have $b=1$, which gives us our original partition function, and it is associative, hence it's a monoid (or a semigroup, depending on your preference of words).
Remark. Observe that $\delta\mathcal{L}(\phi)\sim\mathcal{O}(\lambda)$ which are corrections compensating for removal of large-$k$ components of $\phi$. (End of Remark)
How to get the Callan-Symanzik equation
This is incredibly handwavy. Like, summarizing the story of a three-day relationship between a 13-year old and a 17-year old resulting in 6 deaths (i.e., Romeo and Juliet) level of handwaviness.
So don't follow what I say as the gospel, it's just meant to give some intuition as to what's going on.
Basically, consider the same model, and derive the $n$-point correlation function from the perturbation series. It stands to reason under these circumstances it should be independent of the choice scale. When we impose these symmetry conditions, then consider an "infinitesimal change" in scale, we get a differential equation.
But we demanded no change! So this differential equation (describing the infinitesimal change) should vanish.
Remark. An adequate derivation would have required many pages of manipulation. (That's what I tried doing last night.) So be forewarned, this is just one intuition of what's going on with the renormalization group. There are others; see, e.g., Ticciati's Quantum Field Theory for Mathematicians. For a more detailed explanation/derivation, see Peskin and Schroeder's Introduction to Quantum Field Theory. (End of Remark)
References
Manfred Salmhofer's Renormalization (1999), pg 63 et seq.
Giuseppe Benfatto's Renormalization Group (1995) pg 95 et seq.
Leo P. Kadanoff's University of Chicago course Lecture Slides
N Singh's "Thermodynamical Phase transitions, the mean-field theories, and the renormalization (semi)group: A pedagogical introduction". arXiv:1402.6837
Janos Polonyi's "Lectures on the functional renormalization group method". CEJP 1 (2003) pp 1–71; eprint
Giuseppe Iurato's "On Wilson’s Renormalization Group Structure".
International Journal of Algebra 6 no. 23 (2012) 1121 - 1125; eprint
Lubos Motl's answer to the post "Noether theorem with semigroup of symmetry instead of group"
Best Answer
Schwartz is simply noting that the $\beta$-function has a generic expansion in QED of the form (29) where $\beta_{0,1,2}$ are some numbers that can be computed by explicitly calculating the various Feynman diagrams.
For instance the leading $\epsilon/2$ is the tree-level result in $d=4-\epsilon$ dimensions. This can be easily seen as follows. In $d=4-\epsilon$, we have $[e]=\epsilon/2$ (do you know how to see this??). Then, we find $[\alpha] = [e^2] = \epsilon$. Thus, we can define the dimensionless renormalized coupling $$ \alpha(\mu) = \mu^{-\epsilon} \alpha $$ This is of course a tree-level result as generically $\alpha$ obtains an anomalous dimension. From this equation, we can determine $$ \beta(\alpha) = \mu \frac{d\alpha}{d\mu} = - \epsilon \alpha $$ This is precisely the first term that he has in the $\beta$-function expansion. If I then take into consideration quantum renormalization effects the $\beta$-function takes the more general form (29).
Good! Now, we understand (29). From (29), we can derive (30). Keeping terms to leading order in $\alpha$ in $d=4\implies \epsilon=0$, we find $$ \beta (\alpha) = \mu \frac{d\alpha}{d\mu} = - \frac{\alpha^2}{2\pi} \beta_0 $$ This differential equation can be solved. Doing so introduces an integration constant $\Lambda_{QED}$ and the solution is of the form $$ \alpha(\mu) = \frac{2\pi}{\beta_0} \frac{1}{\log \frac{\mu}{\Lambda_{QED}}} $$ We can now interpret $\Lambda_{QED}$ as the energy scale where $\alpha(\mu) \to \infty$, i.e. it is the scale where the UV description of the theory breaks down. It is known as the Landau pole.