EDIT4: I think I was now able to track down where this dogma originally came from. Howard
Georgi wrote in TOWARDS
A
GRAND
UNIFIED
THEORY
OF
FLAVOR
There is a deeper reason to require the fermion
representation to be complex with respect to SU(3) × SU(2) ×
U(1). I am assuming that the grand unifying symmetry is
broken all the way down to SU(3) × SU(2) × U(1) at a
momentum scale of $10^{15}$ GeV. I would therefore expect any
subset of the LH fermion representation which is real with
respect to SU(3) X SU(2) X U(1) to get a mass of the
order of $10^{15}$ GeV from the interactions which cause the
spontaneous breakdown. As a trivial example of this, consider
an SU(5) theory in which the LH fermions are a 10, a 5
and two $\bar 5$'s. In this theory there will be SU(3) × SU(2) X
U(1) invariant mass terms connecting the 5 to some linear
combination of the two $\bar 5$-'s. These ten (chiral) states will
therefore correspond to 5 four-component fermions with masses
of order 10 as GeV. The 10 and the orthogonal linear
combination of the two 5-'s will be left over as ordinary
mass particles because they carry chiral SU(2) X U(1).
Unfortunately I'm not able to put this argument in mathemtical terms. How exactly does the new, invariant mass term, combining the $5$ and the $\bar 5$ look like?
EDIT3: My current experience with this topic is summarized in chapter 5.1 of this thesis:
Furthermore the group should
have complex representations necessary to accommodate the SU(3) complex triplet
and the complex doublet fermion representation. […] the next five do not have complex representations, and so, are ruled out as candidates for the GUT group. […] It should be pointed out that it is possible to
construct GUT's with fermions in the real representation provided we allow extra
mirror fermions in the theory.
What? Groups without complex representations are ruled out. And a few sentences later everything seems okay with such groups, as long as we allow some extra particles called mirror fermions.
In almost every document about GUTs it is claimed that we need complex representations (=chiral representations) in order to be able to reproduce the standard model. Unfortunately almost everyone seems to have a different reason for this and none seems fully satisfactory to me. For example:
Witten says:
Of the five exceptional Lie groups, four (
G
2
,
F
4
,
E
7
, and
E
8
) only have real or pseu-doreal representations. A four-dimensional GUT model based on such a group will not
give the observed chiral structure of weak interactions. The one exceptional group that
does have complex or chiral representations is E6
This author writes:
Since they do not have complex representations. That we must have
complex representations for fermions, because in the S.M. the fermions
are not equivalent to their complex conjugates.
Another author writes:
Secondly,
the
representations
must
allow
for
the
correct
reproduction
of
the
particle
content
of
the
observed
fermion
spectrum,
at
least
for
one
generation
of
fermions.
This
requirement
implies
that
G
gut
must
possess
complex
representations
as
well
as
it
must
be
free
from
anomalies
in
order
not
to
spoil
the
renormalizability
of
the
grand
unified
theory
by
an
incompatibility
of
regularization
and
gauge
invariance.
The
requirement
of
complex
fermion
representations
is
based
on
the
fact
that
embedding
the
known
fermions
in
real
representations
leads
to
diculties:
Mirror
fermions
must
be
added
which
must
be
very
heavy
.
But
then
the
conventional
fermions
would
in
general
get
masses
of
order
M
gut
.
Hence
all
light
fermions
should
be
components
of
a
complex
representation
of
G
gut
.
And Lubos has an answer that does not make any sense to me:
However, there is a key condition here. The groups must admit complex
representations – representations in which the generic elements of the
group cannot be written as real matrices. Why? It's because the
2-component spinors of the Lorentz group are a complex representation,
too. If we tensor-multiply it by a real representation of the
Yang-Mills group, we would still obtain a complex representation but
the number of its components would be doubled. Because of the real
factor, such multiplets would always automatically include the
left-handed and right-handed fermions with the same Yang-Mills
charges!
So… what is the problem with real representations? Unobserved mirror fermions? The difference of particles and antiparticles? Or the chiral structure of the standard model?
EDIT:
I just learned that there are serious GUT models that use groups that do not have complex representations. For example, this review by Langacker mentions several models based on $E_8$. This confuses me even more. On the one hand, almost everyone seems to agree that we need complex representations and on the other hand there are models that work with real representations. If there is a really good why we need complex representations, wouldn't an expert like Langacker regard models that start with some real representation as non-sense?
EDIT2:
Here Stech presents another argument
The groups E7 and E8 also give rise to vector-like models with $\sin^2 \theta = 3/4$. The mathematical reason is that these groups have, like G and F4, only real (pseudoreal) representations. The only exceptional group with complex… […] Since E7 and Es give rise to vector-like theories, as was mentioned above, at least half of the corresponding states must be removed or shifted to very high energies by some unknown mechanism
Best Answer
Charge conjugation is extremely slippery because there are two different versions of it; there have been many questions on this site mixing them up (1, 2, 3, 4, 5, 6, 7, 8, 9), several asked by myself a few years ago. In particular there are a couple arguments in comments above where people are talking past each other for precisely this reason.
I believe the current answer falls into one of the common misconceptions. I'll give as explicit of an example as possible, attempting to make a 'Rosetta stone' for issues about chirality, helicity, and $\hat{C}$. Other discrete symmetries are addressed here.
A hypercharge example
For simplicity, let's consider hypercharge in the Standard Model, and only look at the neutrino, which we suppose has a sterile partner. For a given momentum there are four neutrino states:
$$|\nu, +\rangle \text{ has positive helicity and hypercharge } Y=0$$ $$|\nu, -\rangle \text{ has negative helicity and hypercharge } Y=-1/2$$ $$|\bar{\nu}, +\rangle \text{ has positive helicity and hypercharge } Y=1/2$$ $$|\bar{\nu}, -\rangle \text{ has negative helicity and hypercharge } Y=0$$
There are two neutrino fields: $$\nu_L \text{ is left chiral, has hypercharge } -1/2, \text{annihilates } |\nu, -\rangle \text{ and creates } |\bar{\nu}, + \rangle$$ $$\nu_R \text{ is right chiral, has hypercharge } 0, \text{annihilates } |\nu, +\rangle \text{ and creates } |\bar{\nu}, - \rangle$$ The logic here is the following: suppose a classical field transforms under a representation $R$ of an internal symmetry group. Then upon quantization, it will annihilate particles transforming under $R$ and create particles transforming under the conjugate representation $R^*$.
The spacetime symmetries are more complicated because particles transform under the Poincare group and hence have helicity, while fields transform under the Lorentz group and hence have chirality. In general, a quantized right-chiral field annihilates a positive-helicity particle. Sometimes, the two notions "right-chiral" and "positive-helicity" are both called "right-handed", so a right-handed field annihilates a right-handed particle. I'll avoid this terminology to avoid mixing up chirality and helicity.
Two definitions of charge conjugation
Note that both the particle states and the fields transform in representations of $U(1)_Y$. So there are two distinct notions of charge conjugation, one which acts on particles, and one which acts on fields. Acting on particles, there is a charge conjugation operator $\hat{C} $ satisfying $$\hat{C} |\nu, \pm \rangle = |\bar{\nu}, \pm \rangle.$$ This operator keeps all spacetime quantum numbers the same; it does not change the spin or the momentum and hence doesn't change the helicity. It is important to note that particle charge conjugation does not always conjugate internal quantum numbers, as one can see in this simple example. This is only true when $\hat{C}$ is a symmetry of the theory, $[\hat{C}, \hat{H}] = 0$.
Furthermore, if we didn't have the sterile partner, we would have only the degrees of freedom created or destroyed by the $\nu_L$ field, and there would be no way of defining $\hat{C}$ consistent with the definition above. In other words, particle charge conjugation is not always even defined, though it is with the sterile partner.
There is another notion of charge conjugation, which on classical fields is simply complex conjugation, $\nu_L \to \nu_L^*$. By the definition of a conjugate representation, this conjugates all of the representations the field transforms under, i.e. it flips $Y$ to $-Y$ and flips the chirality. This is true whether the theory is $\hat{C}$-symmetric or not. For convenience we usually define $$\nu_L^c = C \nu_L^*$$ where $C$ is a matrix which just puts the components of $\nu_L^*$ into the standard order, purely for convenience. (Sometimes this matrix is called charge conjugation as well.)
In any case, this means $\nu_L^c$ is right-chiral and has hypercharge $1/2$, so $$\nu_L^c \text{ is right chiral, has hypercharge } 1/2, \text{annihilates } |\bar{\nu}, +\rangle \text{ and creates } |\nu, - \rangle.$$ The importance of this result is that charge conjugation of fields does not give additional particles. It only swaps what the field creates and what it annihilates. This is why, for instance, a Majorana particle theory can have a Lagrangian written in terms of left-chiral fields, or in terms of right-chiral fields. Both give the same particles; it is just a trivial change of notation.
(For completeness, we note that there's also a third possible definition of charge conjugation: you could modify the particle charge conjugation above, imposing the additional demand that all internal quantum numbers be flipped. Indeed, many quantum field theory courses start with a definition like this. But this stringent definition of particle charge conjugation means that it cannot be defined even with a sterile neutrino, which means that the rest of the discussion below is moot. This is a common issue with symmetries: often the intuitive properties you want just can't be simultaneously satisfied. Your choices are either to just give up defining the symmetry or give up on some of the properties.)
Inconsistencies between the definitions
The existing answer has mixed up these two notions of charge conjugation, because it assumes that charge conjugation gives new particles (true only for particle charge conjugation) while reversing all quantum numbers (true only for field charge conjugation). If you consistently use one or the other, the argument doesn't work.
A confusing point is that the particle $\hat{C}$ operator, in words, simply maps particles to antiparticles. If you think antimatter is defined by having the opposite (internal) quantum numbers to matter, then $\hat{C}$ must reverse these quantum numbers. However, this naive definition only works for $\hat{C}$-symmetric theories, and we're explicitly dealing with theories that aren't $\hat{C}$-symmetric.
One way of thinking about the difference is that, in terms of the representation content alone, and for a $\hat{C}$-symmetric theory only, the particle charge conjugation is the same as field charge conjugation followed by a parity transformation. This leads to a lot of disputes where people say "no, your $\hat{C}$ has an extra parity transformation in it!"
For completeness, note that one can define both these notions of charge conjugation in first quantization, where we think of the field as a wavefunction for a single particle. This causes a great deal of confusion because it makes people mix up particle and field notions, when they should be strongly conceptually separated. There is also a confusing sign issue because some of these first-quantized solutions correspond to holes in second quantization, reversing most quantum numbers (see my answer here for more details). In general I don't think one should speak of the "chirality of a particle" or the "helicity of a field" at all; the first-quantized picture is worse than useless.
Why two definitions?
Now one might wonder why we want two different notions of charge conjugation. Charge conjugation on particles only turns particles into antiparticles. This is sensible because we don't want to change what's going on in spacetime; we just turn particles into antiparticles while keeping them moving the same way.
On the other hand, charge conjugation on fields conjugates all representations, including the Lorentz representation. Why is this useful? When we work with fields we typically want to write a Lagrangian, and Lagrangians must be scalars under Lorentz transformations, $U(1)_Y$ transformations, and absolutely everything else. Thus it's useful to conjugate everything because, e.g. we know for sure that $\nu_L^c \nu_L$ could be an acceptable Lagrangian term, as long as we contract all the implicit indices appropriately. This is, of course, the Majorana mass term.
Answering the question
Now let me answer the actual question. By the Coleman-Mandula theorem, internal and spacetime symmetries are independent. In particular, when we talk about, say, a set of fields transforming as a $10$ in the $SU(5)$ GUT, these fields must all have the same Lorentz transformation properties. Thus it is customary to write all matter fields in terms of left-chiral Weyl spinors. As stated above, this does nothing to the particles, it's just a useful way to organize the fields.
Therefore, we should build our GUT using fields like $\nu_L$ and $\nu_R^c$ where $$\nu_R^c \text{ is left chiral, has hypercharge } 0.$$ What would it have looked like if our theory were not chiral? Then $|\nu, + \rangle$ should have the same hypercharge as $|\nu, -\rangle$, which implies that $\nu_R$ should have hypercharge $-1/2$ like $\nu_L$. Then our ingredients would be $$\nu_L \text{ has hypercharge } -1/2, \quad \nu_R^c \text{ has hypercharge } 1/2.$$ In particular, note that the hypercharges come in an opposite pair.
Now let's suppose that our matter fields form a real representation $R$ of the GUT gauge group $G$. Spontaneous symmetry breaking takes place, reducing the gauge group to that of the Standard Model $G'$. Hence the representation $R$ decomposes, $$R = R_1 \oplus R_2 \oplus \ldots$$ where the $R_i$ are representations of $G'$. Since $R$ is real, if $R_i$ is present in the decomposition, then its conjugate $R_i^*$ must also be present. That's the crucial step.
Specifically, for every left-chiral field with hypercharge $Y$, there is another left-chiral field with hypercharge $-Y$, which is equivalent to a right-chiral field with hypercharge $Y$. Thus left-chiral and right-chiral fields come in pairs, with the exact same transformations under $G'$. Equivalently, every particle has an opposite-helicity partner with the same transformation under $G'$. That is what we mean when we say the theory is not chiral.
To fix this, we can hypothesize all of the unwanted "mirror fermions" are very heavy. As stated in the other answer, there's no reason for this to be the case. If it were, we run into a naturalness problem just as for the Higgs: since there is nothing distinguishing fermions from mirror fermions, from the standpoint of symmetries, there is nothing preventing matter from acquiring the same huge mass. This is regarded as very strong evidence against such theories; some say that for this reason, theories with mirror fermions are outright ruled out. For example, the $E_8$ theory heavily promoted in the press has exactly this problem; the theory can't be chiral.