Expansion is a complicated area of TeX programming. I'll try to explain the key primitives involved first, then try to come up with some examples.
The \expandafter
primitive expands the token after the next one. So
\expandafter\def\csname an-awkward-name\endcsname
will expand \csname
before \def
. So after one expansion the above turns into
\def\an-awkward-name
which will then do its thing. Life becomes more complex when you want to step further ahead, and it soon becomes very hard to track what is going on.
The \edef
> primitive does a full expansion of what is given as its argument (in contrast to \def
, which simply stores the input). So
\def\examplea{more stuff}
\edef\exampleb{Some stuff \csname examplea\endcsname}
will expand the \csname name\endcsname
to \examplea
, then expand that to leave a final definition of \exampleb
as 'Some stuff more stuff'.
Now, \noexpand
comes in by preventing \edef
from doing an expansion of the next token. So if I modify my above example to read
\def\examplea{more stuff}
\edef\exampleb{Some stuff \expandafter\noexpand\csname examplea\endcsname}
then what will happen is that the \edef
will execute the \expandafter
, which will turn the above effectively into
\def\examplea{more stuff}
\edef\exampleb{Some stuff \noexpand\examplea}
Now the \noexpand
will operate (disappearing in the process), leaving the definition of \exampleb
as 'Some stuff \examplea'.
We can use this ability to cut down on \expandafter
use, but there are a couple of other things to know. First, e-TeX includes an additional primitive \unexpanded
, which will prevent expansion of multiple tokens. Secondly, there are various special cases where you don't need quite so many \expandafter
statements. A classic example is from within \csname
, as this will do expansion anyway. So you'll see things like
\csname name\expandafter\endcsname\token
which will expand \token
before \name
.
Back to your example. In the first one, there isn't much to do: as the entire point is to have a dynamic name (#1
), doing an \edef
at point-of-definition doesn't really make sense. The closest one can get is something like
\edef\cohtheory{%
\noexpand\newcommand\expandafter\noexpand\csname foofunc\endcsname[1][*]{%
\noexpand\MakeUppercase{foo}^{##1}}%
}
What will happen here is that \newcommand
and \MakeUppercase
will be protected from expansion, and the \csname
will only expand once. (Tokens which don't have an expansion don't need protection, which is why things like '[1]' are simply included as is.) Of course, this is something of a 'toy' as all it does is create a fixed \foofunc
.
For your second example, you could instead to
\begingroup
\edef\temp{%
\endgroup
\noexpand\command
{\unexpanded\expandafter{\argone}}%
{\unexpanded\expandafter{\argtwo}}%
}
\temp
I'm using a couple of extra ideas here. First, the group is used so that \temp
is not altered anywhere other than where I'm using it. The \endgroup
primitive will do nothing inside the \edef
, and so will still be there to close the group when \temp
is used. Secondly, \unexpanded
works like a toks, and so will respect the \expandafter
after it but before the {
. This cuts down on an unnecessary \expandafter
.
There are more wrinkles to this, and often there are several equally-efficient and clear methods. You are best off posting specific examples, and seeking advice on how they might be achieved.
The key concept here is that, when TeX handles its input, it is doing two distinct things, called expanding and executing stuff. Normally, these activities are interleaved: TeX takes a token (ie, an elementary piece of input), expands it, then executes it (if possible). Then it does so with the next token. But in certain circumstances, most notably when writing to a file, TeX only expands things without executing them (the result will most probably be (re-expanded and) executed later when TeX reads the file back). Some macros, for proper operation, rely on something being properly executed before the next token is expanded. Those are called "fragile", since they work only in the normal (interleaved) mode, but not in expansion-only contexts (such as "moving arguments" which often means writing to a file).
That's the general picture. Now let's give a "few" more details. Feel free to skip to "what to do in practice" :)
Expansion vs execution
The distinction between expansion and execution is somewhat arbitrary, but as a rule of thumb:
- expansion changes only the input stream, ie "what TeX is going to read next";
- execution is everything else.
For example, macros are expandable (TeX is going to read their replacement text next), \input
is expandable (TeX is going to read the given file next), etc. \def
is not expandable (it changes the meaning of the defined macro), \kern
is not expandable (it changes the content of the current paragraph or page), etc.
How things can go wrong
Now, consider a macro \foo
:
\newcommand\foo[1]{\def\arg{#1}\ifx\arg\empty T\else F\fi}
In normal context, \foo{}
gives T
and foo{stuff}
gives F
.) In normal context, TeX will try to expand \def
(which does nothing) then execute it (which removes \arg{#1}
from the input stream and defines \arg
) then expand the next token \ifx
(which removes \arg\empty
and possibly everything up to, but not including, the matching \else
from the input stream), etc.
In expansion-only context, TeX will try to expand \def
(does nothing), then expand whatever comes next ie the \arg
. At this point, anything could happen. Maybe \arg
is not defined and you get a (confusing) error message. Maybe it is defined to something like abc
, so \foo{}
will expand to \def abc{} F
. You'll not get an error when writing this to the file, but it will crash when read back. Perhaps \arg
is defined to \abc
, then \foo{}
will expand to \def\abc{} F
. Then you get no error message either when writing nor at readback, but not only you get F
while you're expecting T
, but also \abc
is redefined, which can have all kinds of consequences if this is an important macro (and good luck for tracking the bug down).
How protection works
Edited to add (not in the original question, but someone asked in a comment): so how does \protect
works? Well, in normal context \protect
expands to \relax
which does nothing. When a LaTeX (not TeX) command is about to process one of its arguments in expansion-only mode, it changes \protect
to mean something based on \noexpand
, which avoids expansion of the next token, thus protecting it from being expanded-but-not-executed. (See 11.4 in source2e.pdf for full details.)
For example, with \foo
as above, if you try \section{\foo{}}
chaos ensues as explained above. Now if you do \section{\protect\foo{}}
then when LaTeX prints the section title it's in normal (interleaved) mode, \protect
expands to \relax
, then \foo{}
expands-and-executes normally and you get a big T in your document. Before LaTeX writes your section title to the .aux
file for the table of contents, it changes \protect
to \noexpand\protect\noexpand
, so \protect\foo
expands to \noexpand\protect\noexpand\foo
and \protect\foo
is written to the aux file. When that line of the aux file is moved to the toc file, LaTeX defines \protect
to \noexpand
, so just \foo
gets written to the toc file. When the toc file is finally read in normal mode, then and only then \foo
is expanded-and-executed and you get a T in your document again.
You can play with the following document, looking at the contents of the .aux
and .toc
files without and with \protect
. Notes: (1) you want to run pdflatex
manually on the file, as opposed to latexmk
or your IDE which might do multiple runs at once, and (2) you will need to remove the toc file to recover after trying the non-\protect
ed version.
\documentclass{article}
\newcommand\foo[1]{\def\arg{#1}\ifx\arg\empty T\else F\fi}
\begin{document}
\tableofcontents
\section{\foo{}} % first run writes garbage to the aux file, second crashes
%\section{\protect\foo{}} % this is fine
\end{document}
Fun fact: the unprotected version fails in a different way (as explained above) if we replace every occurrence of \arg
with \lol
in the definition of \foo
.
Which macros are fragile
This was the easy (read: TeXnical, but well-defined) part of your question. Now, the hard part: when to use \protect
? Well it depends. You cannot know whether a macro is fragile or not without looking at is implementation. For example, the \foo
macro above could use an expandable trick to test for emptyness and would not be fragile. Also, some macros are "self-\protect
ing" (those defined with \DeclareRobustCommand
for example). As Joseph mentioned, \(
is fragile unless you (or another package) loaded fixltx2e
. (As a rule of thumb, most mathmode macros are fragile.) Also, you cannot know whether a particular macro tries to expand-only its arguments, but you can at least be sure all moving arguments will be expanded-only at some point.
What to do in practice
So, my advice is: when you see a weird error happening in or near a moving argument (ie a piece of text that's moved to another part of the document, like a footnote (to the bottom of the page), a section title (to the table of contents), etc), try \protect
ing every macro in it. It solves 99% of the problems.
(This can make you a hero when applied to a colleague's article, due today and "mysteriously" crashing: look at their document for a few seconds before you see a math formula inside a \section
title, say "add a \protect
here", then go back to work and let them call you a wizard. Cheap trick, but works.)
Best Answer
What is
\null
and when do we need to use it?\null
is just\hbox{}
. When is it needed? Never, actually.Why do
\hrulefill
and\hfill
need starting\null
when they follow\\
but not need\null
when they follow\par
?\hrulefill
is\leavevmode\leaders\hrule\hfill\kern0pt
, so it's not exactly like\hfill
, as it adds a null kern after it, but very similar to it. The difference between\par\hrulefill
and\\\hrulefill
is that after a\par
there's a box (the indent box, even with\noindent
), but after\\
there's no box on the line, so glue disappears (and leaders, also).Why does
\hfill
always need ending\null
but\hrulefill
not need it?Because of the
\kern0pt
at the end of\hrulefill
.Why is the first line after
\begin{document}
not regarded as a paragraph recall that starting\null
is needed by\hfill
there?Could you be more precise about this?