[Tex/LaTex] Does \noexpand have to be a primitive

expansiontex-core

Background: I'm trying to deepen my understanding of TeX and how the intricacies work (spurred a little by this place, I should say, and hoping one day to not be a "Cargo Cult Programmer" – see When to use \edef, \noexpand, and \expandafter? both for my other attempts to deepen my knowledge and the – justified – accusation of being a CCP). My source is TeX-by-Topic, but I'm not good at reading from first to last and dip in and out so don't always understand things fully when I read them.

I feel I'm beginning to get the idea of expansion, but still a few things elude me. To try to make an answerable question out of this, let me focus on \noexpand. According to TeX-by-Topic, this is expandable and sets the next token to temporarily be \relax. It's the "temporarily" that I'm finding hard to get my head around. If one wanted to emulate \noexpand as a macro, then the best I could think of was:

\def\noexpand#1{\let\tempa#1%
\let#1\relax
#1%
\let#1\tempa}

but I suspect that this would cause problems in an "expand-until-no-expandables-left" context (such as following an \if). So, to make this a definite question: Can \noexpand be implemented as a macro, or is there some black magic that requires it to be a primitive?

(The picture that I have in mind when I think of \noexpand is that TeX goes in to a ravenous eating mode wherein it chomps up a load of input in its mouth and grinds it up as much as possible before swallowing it, but \noexpand wraps its stuff in an unchewable coating. However, that coating is not immune to the stomach acids. Unfortunately, sometimes TeX (not being well brought up) does a little belch and some of what it ate comes back up to be reconsumed. At this point, the stuff previously protected by \noexpand has had its coating removed by the acids and is now chewable.)

Best Answer

(Further updated answer, with a look deep down the Lion's Mouth in the last section.)

Joseph gave an answer (“No!”) to your actual question; I'll try to get your (and my) head around the “temporarily”. Let's start from TeX-by-Topic, bottom of page 129:

The \noexpand command is expandable, and its expansion is the following token. The meaning of that token is made temporarily equal to \relax, so that it cannot be expanded further.

At first I dismissed this as a colourful description, but one can easily see that TeX-by-Topic is right: The following short plain TeX file

\def\foo{bar}
\expandafter\show\noexpand\foo
\bye

prints

> \foo=\relax.
<recently read> \notexpanded: \foo

to the terminal, so \foo is indeed “made temporarily equal to \relax”. What I find very interesting is the \notexpanded: \foo; this \notexpanded isn't mentioned in the TeXbook or in TeX-by-Topic, and I only found one occurrence in Knuth's torture test for TeX. (Another test is to put \tracingcommands=2 \noexpand\foo into your TeX file; then you'll find a \relax in your log file.)

In the TeXbook there's a description that I find easier chewable than the one from TeX-by-Topic (since it doesn't contain “temporarily”): On page 214 it says

\noexpand⟨token⟩. The expansion is the token itself; but that token is interpreted as if its meaning were ‘\relax’ if it is a control sequence that would ordinarily be expanded by TeX’s expansion rules.

So what does “temporarily” mean?

In practise, “temporarily equal to \relax” means the following: When TeX expands \noexpand\foo, the result is \foo with the temporary meaning \relax, and as soon as TeX continues expanding, \foo gets back its original meaning. A typical use case of \noexpand, from Tex-by-Topic (page 130):

\edef\one{\def\noexpand\two{\the\prevdepth}}

Inside the \edef, TeX doesn't expand \two but continues with the next token { (which is also not expandable). The \noexpand is needed since otherwise TeX would try to expand \two. This would cause an error if \two is undefined, and it would cause much more trouble if \two has been defined before. (Before \def one doesn't need a \noexpand since \def is not expandable.) If you now \show the \one, then you see that the real \two, not a \relaxed one is inside \one:

> \one=macro:
->\def \two {-1000.0pt}.

What's happening behind the scenes?

Let me first say this in the picture of TeX's mouth and stomach. In general, TeX continues chewing on a token list in its mouth as long as the first token is expandable. When the first token is unexpandable, it's swallowed for further processing in the stomach. Now \noexpand indeed wraps the token following it in some protective coating. However, this coating is chewable; figure that it's made of sugar. Thus, when TeX chews on such a coated token, the coating is removed (and the token is expandable again), but TeX gets a sensory flash and thinks “Wow, this tastes like \relax, I want to swallow it.” And sure enough, if the now uncoated token is the first in TeX's mouth (think of “first” as “closest to the throat”), then it is swallowed.

Two examples (the 2nd one being rather academic):

You have, e.g.,
```
\edef\bar{\noexpand\foo\anothercs}
```
Then inside the \edef, \noexpand\foo expands to a \relaxed \foo. This is converted back to \foo, immediately swallowed, and TeX continues with expanding the next token \anothercs.
You expand \noexpand\foo twice, e.g. with
```
\def\foo{bar}
\expandafter\expandafter\expandafter\show\noexpand\foo
\bye
```
Then on the terminal you get
```
> \foo=macro:
->bar.
```
Thus, expanding \noexpand\foo once gives a \relaxed \foo (as seen above), and the expansion of the \relaxed \foo is again the original \foo. And this is not swallowed since it's not the first token!

Disclaimer: The above I found out by observation; it might be that some details are not entirely correct.

Some final remarks

Your attempt at defining \noexpand as macro fails for several reasons. One technical point is that \let#1\relax is not a good idea if #1 isn't a control sequence. Another point is that it's not really helpful to do \let#1\relax #1; this has the same effect as \relax as far as I can tell. (And as pointed out in the other answers, \let is not expandable.)

Related Solutions

[Tex/LaTex] When to use \edef, \noexpand, and \expandafter

Expansion is a complicated area of TeX programming. I'll try to explain the key primitives involved first, then try to come up with some examples.

The \expandafter primitive expands the token after the next one. So

\expandafter\def\csname an-awkward-name\endcsname

will expand \csname before \def. So after one expansion the above turns into

\def\an-awkward-name

which will then do its thing. Life becomes more complex when you want to step further ahead, and it soon becomes very hard to track what is going on.

The \edef> primitive does a full expansion of what is given as its argument (in contrast to \def, which simply stores the input). So

\def\examplea{more stuff}
\edef\exampleb{Some stuff \csname examplea\endcsname}

will expand the \csname name\endcsname to \examplea, then expand that to leave a final definition of \exampleb as 'Some stuff more stuff'.

Now, \noexpand comes in by preventing \edef from doing an expansion of the next token. So if I modify my above example to read

\def\examplea{more stuff}
\edef\exampleb{Some stuff \expandafter\noexpand\csname examplea\endcsname}

then what will happen is that the \edef will execute the \expandafter, which will turn the above effectively into

\def\examplea{more stuff}
\edef\exampleb{Some stuff \noexpand\examplea}

Now the \noexpand will operate (disappearing in the process), leaving the definition of \exampleb as 'Some stuff \examplea'.

We can use this ability to cut down on \expandafter use, but there are a couple of other things to know. First, e-TeX includes an additional primitive \unexpanded, which will prevent expansion of multiple tokens. Secondly, there are various special cases where you don't need quite so many \expandafter statements. A classic example is from within \csname, as this will do expansion anyway. So you'll see things like

\csname name\expandafter\endcsname\token

which will expand \token before \name.

Back to your example. In the first one, there isn't much to do: as the entire point is to have a dynamic name (#1), doing an \edef at point-of-definition doesn't really make sense. The closest one can get is something like

\edef\cohtheory{%
  \noexpand\newcommand\expandafter\noexpand\csname foofunc\endcsname[1][*]{%
  \noexpand\MakeUppercase{foo}^{##1}}%
}

What will happen here is that \newcommand and \MakeUppercase will be protected from expansion, and the \csname will only expand once. (Tokens which don't have an expansion don't need protection, which is why things like '[1]' are simply included as is.) Of course, this is something of a 'toy' as all it does is create a fixed \foofunc.

For your second example, you could instead to

\begingroup
  \edef\temp{%
    \endgroup
    \noexpand\command
    {\unexpanded\expandafter{\argone}}%
    {\unexpanded\expandafter{\argtwo}}%
  }
\temp

I'm using a couple of extra ideas here. First, the group is used so that \temp is not altered anywhere other than where I'm using it. The \endgroup primitive will do nothing inside the \edef, and so will still be there to close the group when \temp is used. Secondly, \unexpanded works like a toks, and so will respect the \expandafter after it but before the {. This cuts down on an unnecessary \expandafter.

There are more wrinkles to this, and often there are several equally-efficient and clear methods. You are best off posting specific examples, and seeking advice on how they might be achieved.

[Tex/LaTex] How to examine a token

Hoping that I understand the question, the options are:

\show, which displays the meaning of a token in the terminal. For registers, this shows the register number rather than the content of the register. Primitives give you an identity, for example \show = \show, whereas for macros you get the expansion.
\showthe, which displays the content of a register at the terminal.
\meaning, which places very similar information in the input stream, with category codes for all tokens 'other' except for spaces. This is useful for both displaying the meaning of macros and for various tests and tokenization tricks. For example, LaTeX uses \meaning to sanitize category codes (although nowadays \scantokens might be a better choice.)
\ifx, which does a comparison based on the meaning of two tokens. The two do not have to be \let to one another, but do have to be identically-defined for the test to be true. So something like
```
\def\testa{stuff}
\def\testb{stuff}
\ifx\testa\testb
```
is true but
```
\def\testa{stuff}
\long\def\testb{stuff}
\ifx\testa\testb
```
is false.
\if, which tests if two unexpandable tokens are identical, ignoring category codes. The two tokens business means that
```
\def\test{aa}
\if\test\SomeOtherThing
```
will compare the two a characters here, and that \SomeOtherThing is part of the true branch.