In principle, the chemical potential in field theories does not differ from the ordinary chemical potential. However, due to the local character of field theories, chemical potentials in field theories have additional properties:
1) In field theories, charges appear as volume integrals of densities:
$$ Q = \int d^3x \rho(x) = \int d^3x J_0(x) $$
(The second expression on the right hand side expresses the fact (by Noether's theorem) that a charge density commuting with the Hamiltonian is the zeroth component of conserved current $J^{\mu} $).
Chemical potentials can also appear as densities thus for a typical chemical potential term:
$$ \mu Q \rightarrow \int d^3x \mu(x) \rho(x) = \int d^3x \mu(x) J^0(x) $$
2) In field theory, currents minimally couple to gauge fields (i.e., through terms of the type $J^{\mu} A_{\mu}$ . Thus the chemical potential term describes a coupling to the zeroth component of a current. Thus it must the zeroth component of an external gauge field. To summarize this point:
Chemical potentials in field theory are the zeroth components of external gauge fields:
$$\mu = A_0$$
3) As in ordinary thermodynamics, in field theory, chemical potentials can be coupled only to conserved charges: $[Q, H] = 0$. For example there exists a Baryon number chemical potential, strangeness chemical potential, etc.
Field theories contain also anomalous symmetries whose corresponding currents are not conserved. However, these currents have non-conservation laws of the type:
$$\partial_{\mu} J^{\mu} = \partial_{\mu} K^{\mu}(A)$$
For example in the case of axial anomaly
$$ K^\mu (x) = -\frac{1}{16\pi^2} \ \varepsilon^{\mu\alpha\beta\gamma}\ t r \ [\frac{1}{2}\ A_\alpha (x)\ \frac{\partial}{\partial x^\beta} \ A_\gamma (x) + \frac{1}{3}\ A_\alpha (x) \, A_\beta (x)\, A_\gamma (x)]$$
The topological current $K^{\mu}$ is not globally defined, however, locally we do have a conserved current:
$$\partial_{\mu} (J^{\mu} - K^{\mu}) = 0$$
It turns out that when an anomalous current is coupled to a chemical potential, one of the most exciting recent discoveries takes place, namely the "Chiral magnetic effect", which consists of a macroscopic manifestation of the chiral anomaly.
This effect can be summarized as follows: In the presence of chiral charge imbalance (thus a nonvanishing chemical potential $\mu_5$ coupled to the axial current), an electric current proportional to the axial current chemical potential is formed in the direction of the external magnetic field.
$$\vec{J} = \frac{e^2}{2 \pi} \mu_5 \vec{B}$$
All the explanations given above (and much more) appear in the following lecture by Kharzeev and references therein.
Best Answer
In statistical mechanics, we always consider systems at thermodynamic limit. By thermodynamic limit, we mean that the volume or the number of particles of a system tends to infinity. It can be shown that the difference between different ensembles vanish at this limit. See, for instance, J. E. Mayer and M. G. Mayer, Statistical Mechanics, (John Wiley, New York, 1940).