Is a latex3 function without a (hollow) star expandable

latex3

In interface3.pdf sec 1.2, it says the function with a star can be fully expandable in x, e, f, and the function with a hollow star can be fully exandable in x and e, and cannot be fully expanded within an f-type argument. My first question is

How deeply will f-type argument expand the "hollow star"

And my main question: there are also some funtion without star, like \tl_set_eq:NN, how will x, e, f-type arguments regard it? I made some test and the output confused me.

\documentclass{article}
\begin{document}
\ExplSyntaxOn
    \tl_new:N \l_tmpc_tl
    \tl_new:N \l_tmpd_tl
    \tl_set:Nn \l_tmpa_tl { abc }
    \tl_set:Nn \l_tmpb_tl { ABC }
    % I want to reset \l_tmpb_tl the same as \l_tmpa_tl, and set \l_tmpc_tl (\l_tmpd_tl)  
    \tl_set:Nf \l_tmpc_tl { \tl_set_eq:NN \l_tmpb_tl \l_tmpa_tl \l_tmpb_tl }
    \tl_set:Nx \l_tmpd_tl { \tl_set_eq:NN \l_tmpb_tl \l_tmpa_tl \l_tmpb_tl }
    \tl_show:N \l_tmpc_tl
    \tl_show:N \l_tmpd_tl
\ExplSyntaxOff
\end{document}

After compiling with pdflatex, it output:

> \l_tmpc_tl=\tex_let:D \l_tmpb_tl = \l_tmpa_tl \l_tmpb_tl .
<recently read> }

l.11     \tl_show:N \l_tmpc_tl

?
> \l_tmpd_tl=\tl_set_eq:NN ABCabcABC.
<recently read> }

l.12     \tl_show:N \l_tmpd_tl

?

We can see that f-type expanded \tl_set_eq:NN \l_tmpb_tl \l_tmpa_tl into a tex primitive:

\tex_let:D \l_tmpb_tl = \l_tmpa_tl \l_tmpb_tl

However x-type didn't do things on \tl_set_eq:NN, it just left it in the output stream. ~~Why would this happen?~~ And how does latex3 expand a no-star-function?

UPDATE:

Countinuing reading interface3.pdf, I found it in sec 4.3:

Functions which are not “protected” are fully expanded inside an x expansion. In contrast, “protected” functions are not expanded within x expansions.

And I reached back to the definition of \tl_set_eq:NN, it is "protected"

> \tl_set_eq:NN=\protected\long macro:#1#2->\tex_let:D #1= #2.
<recently read> }

l.13     \cs_show:N \tl_set_eq:NN

So it can't be expanded by x-type argument, and protected function can be expanded by f-type. Am I right here?

And then my second question becomes: Are all functions without a star protected?

Best Answer

Yes, all functions without a star are (or at least should be) \protected, this way they don't expand on x or e type expansion or when written to a file.

f and c type expansion are a different and ignore \protected. But it is wrong to (intentionally) expand them (the functions without stars) with f-type expansion.

f-type expansion will expand everything until it hits the first unexpandable token (as already said ignoring \protected, so this means until it hits either an unexpandable primitive or a character token of categories 1, 2, 3, 4, 6, 7, 8, 11, or 12) or a space (the space would then be gobbled, the unexpandable token would stay there).

By the way, many of the hollow-starred functions aren't necessarily f-type safe, but could be, this depends on the user code being supplied. For instance \int_step_function:nN is not f-expansion safe, but a usage such as \int_step_function:nN {10} \use_none:n would be (though admittedly this is a nonsense example). The reason for this (the fact they are marked with a hollow star) is usually because they iterate over some input and for each element leave some code in the input stream, and that code could be unexpandable or not whatever, so there is no guarantee that it would always work out.

Related Solutions

[Tex/LaTex] A guide to understanding expandability: when to write protected functions and when not to

An expandable command is one which can be converted 'fully' into it's output inside a TeX \edef or \write (and a few other places). Thus for example

\def\testa{\testb}
\def\testb{\testc}
\def\testc{d}
\edef\teste{\testa}
\show\teste

will give

> \teste=macro:
->d.

i.e. all of the steps have been expanded, and we have just characters.

For text, this is nice and simple, but when you get TeX primitives involved things are more complex as some are expandable and some are not. Broadly, anything which performs an assignment is unexpandable. So if we have

\def\testa{\testb}
\def\testb{\testc}
\def\testc{\def\ARG{d}}
\def\ARG{}
\edef\teste{\testa}
\show\teste

we get

> \teste=macro:
->\def {d}.

Notice how the \def is left unchanged but the \ARG has vanished: it got expanded to what it is defined as (empty).

e-TeX allows us to define a protected macro. These do not expand inside an \edef, so

\def\testa{\testb}
\def\testb{\testc}
\def\testc{\def\NOTARG{d}}
\protected\def\NOTARG{}
\edef\teste{\testa}
\show\teste

now yields

> \teste=macro:
->\def \NOTARG {d}.

There is a subtle but important point here: \def is an unexpandable primitive, while \NOTARG is now a protected macro. You can tell that \NOTARG is protected using \show:

> \NOTARG=\protected macro:
->.

where the \protected tells us what we need to know. However, you have to know that \def is not expandable.

In the LaTeX3 documents, rather than expect people to learn the rules we've gone with a different approach: we document which functions are expandable (they are marked with a star). The reason everything else is then protected is that 'partial' expansion is a real issue. If you do

\def\testa{\let\testb\testc}
\edef\testb{\testa}

you get

! Undefined control sequence.
\testa ->\let \testb 
                     \testc

as \let is unaffected by the \edef but \testb is undefined. This gets worse when you look at 'real' documents, as the problem can be hidden many layers down.

Many of the issues people see in real LaTeX2e documents, for example where they forget \protect and have trouble, would be bypassed if most commands were protected. In general, you find a lot more (La)TeX code that is not expandable than code that is, so the position for LaTeX3 is that this is the exception, certainly for document commands. (Typesetting is not expandable, and that's what happens in documents.)

This leads us on to what I call the 'sheep and goats' approach to protected functions: all LaTeX3 code is either protected or fully expandable ['safe' (will give the expected result) inside \edef/x-type expansion], even if we are talking about auxiliary functions. The result is that we can always be sure if a function can be used in an expansion context: if it can, it's marked with a star, otherwise it will be protected and won't expand part way. So the 'correct' way to write LaTeX3 code is that if you use anything that is not expandable (i.e. not starred in the documentation) in your code, then you have to use \cs_new_protected:Npn or similar, and not \cs_new:Npn, etc.

LaTeX3 how to use content/value of predefined command in token list/string

You have declared \testcommand as a document command: these are protected from expansion. That means that \tl_set:Nx \l_test_tl { \testcommand } doesn't change the result. I would declare \testcommand as an expandable command and use V-type expansion:

\newcommand*\testcommand{A Test Sentence for expl3}
\tl_set:NV \l_test_tl \testcommand

etc.

Best Answer

Related Solutions

[Tex/LaTex] A guide to understanding expandability: when to write protected functions and when not to

LaTeX3 how to use content/value of predefined command in token list/string

Related Question