First of all, Sean Carroll is a relativist so his treatment of the diffeomorphism symmetry as a gauge symmetry should be applauded because it's the standard modern view preferred by particle physicists – its origin is linked to names such as Steven Weinberg, it is promoted by physicists like Nima Arkani-Hamed, and naturally incorporated in string theory so seen as "obvious" by all string theorists. In this sense, Carroll throws away the obsolete "culture" of the relativists. There are some other "relativists" who irrationally whine that it shouldn't be allowed to call the metric tensor "just another gauge field" and the diffeomorphism group as "just another gauge symmetry" even though this is exactly what these concepts are.
Second of all, a symmetry expressed by a Lie algebra can't be "discrete", by definition: it is continuous. Lie groups are continuous groups; it is their definition. And only continuous groups are able to make whole polarizations of particles unphysical. It's plausible that a popular book replaces the continuous groups by discrete ones that are easier to imagine by the laymen but this server is not supposed to be "popular" in this sense.
Third, when you say that if $U$ is unitary, the generator has to be Hermitian and traceless, is partly wrong. Unitarity of $U$ means the hermiticity of the generators $T^a$ but the tracelessness of these generators is a different condition, namely the property that $U$ is "special" (having the determinant equal to one). The tracelessness is what reduces $U(N)$ to $SU(N)$, unitary to special unitary.
Fourth, and it is related to the second point above, "charge conjugation" isn't any gauge principle of electromagnetism in any way. Electromagnetism is based on the continuous $U(1)$ gauge group. This group has an outer automorphism – a group of automorphisms is ${\mathbb Z}_2$ – but we're never putting these elements of the discrete group into an exponent.
Fifth, similarly, QCD isn't based on the discrete symmetry of permutations of the colors but on the continuous $SU(3)$ group of special unitary transformations of the 3-dimensional space of colors. Because none of the things you wrote about the non-gravitational case was quite right, it shouldn't be surprising that you have to encounter lots of apparent contradictions in the case of gravity as well because gravity is indeed more difficult in some sense.
Sixth, $SO(3,1)$ isn't related to the diffeomorphism in any direct way. It is surely not the same thing. This group is the Lorentz group and in the GR, you may choose a formalism based on tetrads/vielbeins/vierbeins where it becomes a local symmetry because the orientation of the tetrad may be rotated by a Lorentz transformation independently at each point of the space. But this is just an extra gauge symmetry that one must add if he works with tetrads – it's a symmetry that exists on top of the diffeomorphism symmetry and this symmetry is different and "non-local" because it changes the spacetime coordinates of objects or fields while all the Yang-Mills symmetries above and even the local Lorentz group at the beginning of this paragraph are acting locally, inside the field space associated with a fixed point of the spacetime. (The fact that diffeomorphisms in no way "boil down" to the local Lorentz group is a rudimentary insight that is misunderstood by all the people who talk about the "graviweak unification" and similar physically flawed projects.) I will not use with tetrads in the next paragraph so the gauge symmetry will be just diffeomorphisms and there won't be any local Lorentz group as a part of the gauge symmetry.
The diffeomorphism symmetry is locally generated by the translations, not Lorentz transformations, and the parameters of these 4-translations depend on the position in the 4-dimensional spacetime. This is how a general infinitesimal diffeomorphism may be written down. If there were no gauge symmetries, $g_{\mu\nu}$ would have 10 off-shell degrees of freedom, like 10 scalar fields. However, each generator makes two polarizations unphysical, just like in the case of QED or QCD above (where the 4 polarizations of a vector were reduced down to 2; in QCD, all these numbers were multiplied by 8, the dimension of the adjoint representation of the gauge group, $SU(3)$ etc.). Because the general translation per point has 4 parameters, one removes $2\times 4 = 8$ polarizations and he is left with $10-8=2$ physical polarizations of the gravitational wave (or graviton). The usual bases chosen in this 2-dimensional physical space is a right-handed circular plus left-handed circular polarized wave; or the "linear" polarizations that stretch and shrink the space in the horizontal/vertical direction plus the wave doing the same in directions rotated by 45 degrees:
This counting was actually a bit cheating but it does work in the general dimension. To do the counting properly and controllably, one has to distinguish constraints from dynamical equations and see how many of the modes of a plane wave (gravitational wave) are affected by a diffeomorphism. In the general dimension of $d$, it may be seen that the tensor $\Delta g_{\mu\nu}$ may be described, after making the right diffeomorphism, by $h_{ij}$ in $d-2$ dimensions and moreover the trace $h_{ii}$ may be set to zero. This gives us $(d-2)(d-1)/2-1$ physical polarizations of the graviton. In $d=4$, this yields 2 physical polarizations of the graviton. A gravitational wave moving in the 3rd direction is described by $h_{11}=-h_{22}$ and $h_{12}=h_{21}$ while other components of $h_{\mu\nu}$ may be either made to vanish by a gauge transformation (diffeomorphism), or they're required to vanish by the equations of motion or constraints linked to the same diffeomorphism. Morally speaking, it is true that we eliminate two groups of 4 degrees of freedom, as I indicated in the sloppy calculation that happened to lead to the right result. Note that
$$\frac{d(d+1)}2 -2d = \frac{(d-2)(d-1)}2-1 $$
I have to emphasize that these is a standard counting of the "linearized gravity" and it's the same procedure to count as the counting of physical polarizations after the diffeomorphism "gauge symmetry" – just the language involving "gauge symmetries" is more particle-physics-oriented.
Looks like you are trying to understand the color charge of the color SU(3) through the flavor electric charge of flavor SU(3), which has nothing-nothing-nothing to do with it, except we use the same hermitian generators, the Gell-Mann matrices halved, for the triplet (quark) representation, to describe them.
In the former case, any flavor of quarks has eight color charges , i = 1, ..., 8,
$$
\int\!\! d^3x~~ \bar q \cdot \tfrac{1}{2}\lambda^i ~\gamma ^0 q,
$$
which are strictly conserved, and are each coupled to the eight respective gluons (the color gauge fields) for consistency of the theory. Their actual values, related among themselves, are immaterial, and can be linearly combined and/or absorbed into redefinitions of them and their gauge-fields. Their eigenvalues hardly matter; for the triplet of quarks, for whose labels people use R,G,B, each gluon mutates color differently, and all you'd need is to ensure total color is conserved in an interaction.
In the latter case, for flavor SU(3), the light, u,d,s, quarks, you have eight almost conserved flavor charges that look like the above ones, but are violated by small amounts, isospin and hypercharge breaking.
It turns out the electric charge Q is related to this quark triplet by the Gell-Mann–Nishijima formula ,
$$
Q= \frac{\lambda^3}{2} +\frac{\sqrt{3}}{6} \lambda^8.
$$
(I simplified it for your convenience, since it modifies trivially for other hadrons.) You may confirm the electric charges (2/3, -1/3,-1/3) of the (u,d,s) triplet: that was the design!
The normalizations are, indeed, irrational and freaky, but the action of the Cartan subalgebra of these two generators (simultaneously diagonal in this basis) "conspires" to yield related eigenvalues in the root lattice, so
- the resulting electric charges are multiples of each other! So the nonabelian flavor group (here SU(3), ungauged)is primed to yield charge quantization at the end of the day.
Best Answer
The elements of the gauge transformations belong to a gauge group. In physics, it's most typically $SU(N)$ (both the electroweak theory, with its $SU(2)$, and the QCD for quarks, $SU(3)$, use these $SU(N)$ groups; $U(1)$ we first learn in electromagnetism – but we must reinterpret the charge as the "hypercharge" when we study the electroweak theory – is the only extra addition we need for the Standard Model). It's a group of all complex $N\times N$ matrices $M$ that obey $$MM^\dagger=1, \quad \det M = 1$$ Note that $M^\dagger=(M^*)^T$ is the Hermitian conjugate; the first condition makes the matrix "unitary", therefore $U$. The determinant of a unitary matrix could be any complex number whose absolute value equals one. The second condition says that the determinant must be one and nothing else, that's the "special" or $S$ condition in $SU(N)$.
The gauge field transforms as $$ A_\mu \to M(A_\mu+ie\partial_\mu) M^\dagger$$ up to different conventions. That's needed for the covariant derivative $D_\mu$ to transform nicely. Forget about the complicated formula above. The point is that $A_\mu$ takes values in the Lie algebra of the Lie group.
In other words, you may imagine an infinitesimal transformation – infinitely close to the identity – in the gauge group, e.g. $SU(N)$. Assume $$ M = 1+i\epsilon G $$ The factor $\epsilon$ makes it infinitesimal, the factor of $i$ is a convention popular among physicists but omitted by mathematicians (physicists like things to be Hermitian, without $i$, they would have to be anti-Hermitian).
Here, $G$ is the kind of $N\times N$ matrix that the gauge field can have as a value.
Now, substitute this Ansatz for $M$ into the conditions $MM^\dagger=1,\det M=1$. You may neglect $\epsilon^2$ "very small" terms and the conditions become $$1+i\epsilon G - i\epsilon G^\dagger = 1, \quad \det(1+i\epsilon G) = 1$$ Mathematics implies that these conditions are equivalent to $$ G = G^\dagger, \quad {\rm Tr}(G) =0 .$$ To get the first one, I just subtracted $1$ from both sides and cancelled $i\epsilon$. To get the latter, I used the "sum of products over permutations" formula for the determinant and noticed that only the product of the diagonal entries contributes $O(\epsilon)$ terms and they're proportional to the sum of the diagonal entries, the trace.
At any rate, you should try to understand this maths and its conclusion is that the Hermiticity of the generator $G$ – matrices that are combined with various real coeffcients to get $A_\mu$ – is equivalent to the gauge group's being unitary; and the tracelessness is equivalent to the group's being "special" i.e. requiring the unit determinant.
It's perhaps useful to mention why $SU(N)$ is considered the "simplest" class of gauge groups. The $S$ has to be there because $U(N)$ isn't simple – it's pretty much isomorphic to $SU(N)\times U(1)$ where the two factors could be treated separately and we want to work with the smallest allowed pieces of gauge groups which are $SU(N)$ and $U(1)$. And $SU(N)$ is more "elementary" than $SO(N)$ or $USp(2N)$ because complex numbers are more fundamental in group theory (and physics) than real numbers or quaternions. In fact, the groups $SO(N)$ and $USp(2N)$ may be defined as $SU(N)$ with some "extra structure" (orientifolds) added which makes some natural group-theoretical analyses somewhat more convoluted than those for $SU(N)$. But one may still say that the Lie algebra for $SO(N)$ would be composed of antisymmetric real (or antisymmetric pure imaginary, depending on the conventions concerning factors of $i$) matrices, in analogy with the Hermitian matrices above; they're automatically traceless.