I'm trying to find a way to use the e-TeX \scantokens
command inside [the equivalent of] an \edef
(well, actually an \xdef
, but presumably this does not make any real difference since that's just the \global
version of \edef
). The reason I am doing this is that it seems to be needed to answer this question about hyperref and pdf outlines.
The problem seems to be that \scantokens
doesn't actually expand to the tokens scanned; it actually expands to nothing, but arranges for the next tokens to be read from a pseudo-file named " ", which is then treated almost exactly like a real file, including the check for "runaway" conditions. (See page 10 of the e-TeX manual.)
There is one problem with this explanation, though: why would \input
work, as it evidently does in the other question?
Anyway, here's a fairly minimal example:
%&eplain
% make \scantokens pseudo-files show up with name " " in the output
\tracingscantokens=1
\edef\foo{\scantokens{Test}}
Which produces the output
Running `TeX' on `scantokens-in-edef-minimal' with ``pdftex -interaction=nonstopmode "\input" "scantokens-in-edef-minimal"'' This is pdfTeX, Version 3.1415926-1.40.11 (MiKTeX 2.9) entering extended mode (e:\home\tex\scantokens-in-edef-minimal.tex ( ) Runaway definition? ->Test ! File ended while scanning definition of \foo. <inserted text> } l.6 \edef\foo{\scantokens{Test} } ! Too many }'s. l.6 \edef\foo{\scantokens{Test}} ) ! Emergency stop. <*> \input scantokens-in-edef-minimal ! ==> Fatal error occurred, no output PDF file produced! Transcript written on scantokens-in-edef-minimal.log. TeX exited abnormally with code 1 at Thu Jan 27 12:24:57
Is it perhaps possible to somehow do the expansion with macros and then do a \def
instead of the \edef
, so the expansion would be finished before the definition began, and the "runaway definition" condition avoided?
Best Answer
In short, yes. Try
or
in which the
\noexpand
‘hides’ the EOF marker by turning it into\relax
.You might like to take a look at the definition of expl3's
\tl_rescan:nn
and\tl_set_rescan:Nnn
, both wrappers for\scantokens
based on the ideas in Heiko Oberdiek'scatchfile
package.Then you can write
or
The latter cannot be used inside an
\edef
but can otherwise come in handy when you're processing a macro argument that has already been tokenised. In practise I find theset_rescan
version a little more convenient.