[Tex/LaTex] How to get a value out of a group

expl3groupinglatex3programming

I've just recovered from my first real foray into LaTeX3 programming. It was … an experience. Not a completely unpleasant one and once I'd gotten used to the syntax then I found it a lot easier than it looks. I almost was able to forget that I was programming in a macro language and start thinking in terms of functions and variables again.

Almost.

Where I came up against a brick wall was in the concept of auxiliary functions. I had a main function that did a lot of work. I would quite like to have farmed off parts of it to some other functions, as much for keeping track of what was going on as anything else. But I couldn't work out how to do that separation properly.

Here's an example. Suppose I wanted to compute the Euclidean length of a lot of 2-vectors. In, say, lua then I might write:

function veclen(a,b)
  return math.sqrt(a^2 + b^2)
end

Now that isn't directly comparable with a LaTeX3 "function" so let me pretend that I can't write inline formulae in lua and write it out a bit more like LaTeX3.

function veclen(a,b)
  local s = a
  local t = b
  multiply(s,s)
  multiply(t,t)
  add(s,t)
  sqrt(s)
  return s
end

Here, multiply and add are functions that "do what they say on the tin" but, crucially, instead of returning a value they store the answer in the first variable.

That's fairly similar to my LaTeX3 function:

\cs_new:Nn \fp_veclen:NNN {
  \fp_set_eq:NN \l_hobby_veclena_fp #2
  \fp_set_eq:NN \l_hobby_veclenb_fp #3
  \fp_mul:Nn \l_hobby_veclena_fp {\l_hobby_veclena_fp}
  \fp_mul:Nn \l_hobby_veclenb_fp {\l_hobby_veclenb_fp}
  \fp_add:Nn \l_hobby_veclena_fp {\l_hobby_veclenb_fp}
  \fp_pow:Nn \l_hobby_veclena_fp {.5}
  \fp_set_eq:NN #1 \l_hobby_veclena_fp
}

The crucial difference, and what I'm asking about, is the scoping.

In the lua function, a and b are local but even if they are not then the commands local s = a and local t = b force them to be local. But the return s breaks out of the function scope and makes the answer available at the next level up. In doing this in TeX, I'd do all my computations inside a group: something like:

\cs_new:Nn \fp_veclen:NNN {
\group_begin:
  \fp_set_eq:NN \l_hobby_veclena_fp #2
  \fp_set_eq:NN \l_hobby_veclenb_fp #3
  \fp_mul:Nn \l_hobby_veclena_fp {\l_hobby_veclena_fp}
  \fp_mul:Nn \l_hobby_veclenb_fp {\l_hobby_veclenb_fp}
  \fp_add:Nn \l_hobby_veclena_fp {\l_hobby_veclenb_fp}
  \fp_pow:Nn \l_hobby_veclena_fp {.5}
\group_end:
\fp_set_eq:NN #1 \l_hobby_veclena_fp
}

Except that that wouldn't work: if I put the \group_end: where I have done so then \l_hobby_veclena_fp has lost its value. If I put it after the assignment then the assignment is local to the group and so is lost moments later.

To cut a long story short: I want the computation to be local so that I can use my temporary variables with impunity, but I need the assignment to be outside the computation group (but not global) so that it can be used by the calling code.

How do I do this? Obviously it is possible because the LaTeX3 functions must do this all the time[1]. TikZ uses "smuggling" to do this: define a temporary global variable to be the answer and then outside the group make the assignment which makes the actual assignment not global but outside the computation group: \global\let\tikz@smuggle=\the@answer\endgroup\let\the@answer=\tikz@smuggle. What's the right LaTeX3-way to do this?

For bonus points, there's another small issue with scoping. In the lua function, the variables a and b were already local: I could reassign them with impunity. In TeX, that's not so easy. If I do something like

\fp_set_eq:NN \l_my_tmpa_fp #1
\fp_set_eq:NN \l_my_tmpb_fp #2

then there's always the danger that I called my function with \calc:NN \l_my_tmpb_fp \l_my_tmpa_fp. How do I make local aliases for my incoming variables without defining a new set of temporary macros for every single function?

[1] I know I could look at the LaTeX3 code – indeed I took a quick glance and got a vague idea, but I suspect that there would be subtleties I'd miss by not asking, and I hope that others will benefit from a more public answer and explanation.

Best Answer

Soln 1

This might be considered overkill, but I once wrote a function \group_after_set:NNn to provide an "abstraction" to handle this, used as in:

\group_begin:
  \group_begin:
    \group_after_set:NNn \int_set:Nn \y {3}
    % \y == 3
  \group_end:
  % \y == 3
\group_end:
% \y == undefined

I quite like it but I don't know if should be added to expl3 or not. Suggestions?

The advantage over this approach is that you can use it multiple times within a group to "export" multiple variables, unlike Enrico's answer which is much more simple and efficient for the common case.

Anyway, here it is:

\documentclass{article}
\usepackage{expl3}
\begin{document}
\ExplSyntaxOn
\cs_if_free:NT \group_insert_after:N
  {
    \cs_set_eq:NN \group_insert_after:N \group_execute_after:N
  }

\cs_generate_variant:Nn \tl_if_empty:nT {v}
\cs_generate_variant:Nn \tl_show:N {v}
\cs_new:Nn \group_after_set:NNn
{
  % set the variable locally for use inside the group:
  #1 #2 {#3}

  \cs_if_exist:cF { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
    {
      \tl_new:c { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
    }

  % first time the function is executed inside the group:
  \tl_if_empty:vT { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
    {
      % set up the aftergroup execution:
      \group_insert_after:c
        { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }

      % reset the material for aftergroup execution:
      \tl_gset:cx { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
        {
          \tl_gclear:c { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
        }
    }

  % append the new material to the aftergroup execution:
  \tl_gput_right:cx
    { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
    {
      \exp_not:n { #1 #2 } { \exp_not:V #2 }
    }
}

\cs_generate_variant:Nn \group_insert_after:N {c}

\int_new:N \y
\int_new:N \yy

\cs_generate_variant:Nn \tl_if_eq:nnF {V}
\cs_new:Nn \assert:Nn
  {
    \tl_if_eq:VnF #1 {#2} {[ERROR:\tl_to_str:n{#1~!=~#2}]}
  }

\group_begin:
  \group_after_set:NNn \tl_set:Nn \x {xx}
  \group_after_set:NNn \tl_set:Nn \xx {x}
  \assert:Nn \x {xx}
  \assert:Nn \xx {x}
  \group_begin:
    \group_after_set:NNn \int_set:Nn \y {3}
    \group_after_set:NNn \int_set:Nn \yy {2}
    \assert:Nn \y  {3}
    \assert:Nn \yy {2}
  \group_end:
  \assert:Nn \x {xx}
  \assert:Nn \xx {x}
  \assert:Nn \y  {3}
  \assert:Nn \yy {2}

  % check repeated doesn't break anything:
  \group_begin:
    \int_set:Nn \y  {6}
    \int_set:Nn \yy {5}
  \group_end:
  \assert:Nn \x {xx}
  \assert:Nn \xx {x}
  \assert:Nn \y  {3}
  \assert:Nn \yy {2}
\group_end:

\assert:Nn \x {xx}
\assert:Nn \xx {x}
\assert:Nn \y  {0}
\assert:Nn \yy {0}

\ExplSyntaxOff
\end{document}

Edit: add some comments and clean up the code slightly.

Soln 2 (added later)

From some discussion elsewhere on this page with Ahmed, it's become clear to me that the syntax introduced about isn't such a great idea. For example, if you want to set a variable using tl_set:Nx, it's not then appropriate to use that same function to define the variable outside the group. Really what we're looking for is an extension of etextool's \AfterGroup that permits easy expansion control (which is part of the whole design philosophy of expl3).

So separating the local and "group escaped" variables need to be done independently, syntax-wise. Here's a modified implementation of the above that does this. The code looks similar:

\int_new:N \y
\group_begin:
  \group_begin:
    \int_set:Nn \y {3}
    \group_var_return:NN \int_set:Nn \y
    % \y == 3
  \group_end:
  % \y == 3
\group_end:
% \y == 0

Note that since we don't have a mapping between variable types and their setting functions, in this case \int_set:Nn is required twice. This isn't so bad, really, since you could in theory use this for all sorts of setting functions. So the "general" approach here I've written as follows:

\clist_new:N \z
% ...

\group_begin:
  % let's say we're in a macro right now that processes
  % some input argument and saves the result to "\y".
  \do_something_with:Nn \y {#1}
  \group_after_insert:nV { \clist_put_right:Nn \z } { \y }
\group_end:

So it's easy to see how the former (\group_var_return:NN) is just something like \group_after_insert:nV { \int_set:Nn \y } \y. Whether these levels of abstraction are necessary or helpful is a bit of an open question. I'm inclined to think that at least the \group_var_return:NN one looks good.

Here is the implementation of all that:

\documentclass{article}
\usepackage{expl3}
\begin{document}
\ExplSyntaxOn

\cs_if_free:NT \group_insert_after:N
  {
    \cs_set_eq:NN \group_insert_after:N \group_execute_after:N
  }

\cs_generate_variant:Nn \tl_if_empty:nT {v}
\cs_generate_variant:Nn \group_insert_after:N {c}

\cs_new:Nn \group_after_insert:nn
{ 
  \cs_if_exist:cF { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
    {
      \tl_new:c { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
    }

  % first time the function is executed inside the group:
  \tl_if_empty:vT { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
    {
      % set up the aftergroup execution:
      \group_insert_after:c
        { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }

      % reset the material for aftergroup execution:
      \tl_gset:cx { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
        {
          \tl_gclear:c { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
        }
    }

  % append the new material to the aftergroup execution:
  \tl_gput_right:cx
    { g_aftergroup_ \int_use:N \etex_currentgrouplevel:D _tl }
    {
      \exp_not:n { #1 {#2} }
    }
}

\cs_generate_variant:Nn \group_after_insert:nn {nV}

\cs_new:Npn \group_var_return:NN #1 #2
{
  \group_after_insert:nV { #1 #2  } { #2 } 
}

\int_new:N \y
\int_new:N \yy

\cs_generate_variant:Nn \tl_if_eq:nnF {V}
\cs_new:Nn \assert:Nn
  {
    \tl_if_eq:VnF #1 {#2} {[ERROR:\tl_to_str:n{#1~!=~#2}]}
  }

\group_begin:
  \tl_set:Nn \x {xx}
  \tl_set:Nn \xx {x}
  \group_after_insert:nV { \tl_set:Nn \x  } { \x  }
  \group_after_insert:nV { \tl_set:Nn \xx } { \xx }
  \assert:Nn \x {xx}
  \assert:Nn \xx {x}
  \group_begin:
    \int_set:Nn \y {3}
    \int_set:Nn \yy {2}
    \group_var_return:NN \int_set:Nn \y
    \group_var_return:NN \int_set:Nn \yy
    \assert:Nn \y  {3}
    \assert:Nn \yy {2}
  \group_end:
  \assert:Nn \x {xx}
  \assert:Nn \xx {x}
  \assert:Nn \y  {3}
  \assert:Nn \yy {2}

  % check repeated doesn't break anything:
  \group_begin:
    \int_set:Nn \y  {6}
    \int_set:Nn \yy {5}
  \group_end:
  \assert:Nn \x {xx}
  \assert:Nn \xx {x}
  \assert:Nn \y  {3}
  \assert:Nn \yy {2}
\group_end:

\assert:Nn \x {xx}
\assert:Nn \xx {x}
\assert:Nn \y  {0}
\assert:Nn \yy {0}

\ExplSyntaxOff
\end{document}

Related Solutions

[Tex/LaTex] Why are box dimension assignments both local and global

I am just guessing here, but I believe box dimensions are associated with each box. You don't have special dimen registers for these. This is what I believe happens here. After you enter the first group, at the moment you assign \setbox0\hbox{ZZZ}, TeX assigns a local box register. You then enter the second group, but your box register 0 is still the same \hbox{ZZZ}, TeX will not create a local copy of the box. So when you assign 0pt to \wd0, you modify the \hbox{ZZZ} from the previous group.

Try to modify your code like this:

\setbox0 \hbox{XXX}
\fbox{\copy0}

{
   \wd0 0pt
   \setbox0\hbox{ZZZ}
   {\wd0 0pt}
   \fbox{\copy0}
}

\fbox{\box0}

and see what happens.

Edit: I think the following behavior is related to this:

\setbox0 \hbox{XXX}
\fbox{\copy0}

{
   \fbox{\box0}
}

\fbox{\box0}

Notice that the \box0 inside the group empties the box register, it does not get restored at the end of the group.

[Tex/LaTex] LaTeX3 versus pure Lua

(I guess I could be called a member of one of the teams ;-) this is my view)

I thought of staying out of this debate, but perhaps some words of clarification or, let's say, some thoughts are in order after all.

LaTeX3 versus pure Lua

First of all this is the wrong question imho: LaTeX3 has different goals to LuaTeX and those goals may well be still a defunct pipe dream, but if so they are unlikely to be resolved by pure Lua either.

So if one wants to develop an argument along those lines then it should be more like "Why does LaTeX3 use an underlying programming language based on eTeX and not on LuaTeX where a lot of functionality would be available in a "simpler" way?"

But LaTeX3 is really about three to four different levels

underlying engine
programming level
typesetting element layer
designer interface foundation layer
document representation layer

See for example my talk at TUG 2011: http://www.latex-project.org/papers/

Here is a sketch of the architecture structure:

enter image description here

The chosen underlying engine (at the moment is any TeX engine with e-TeX extension). The programming level is what we call "expl3" and that is what I guess you are referring to if you say "LaTeX3" (and I sometimes do that too). However, it is only the bottom box in the above diagram. The more interesting parts are those that are above the programming level (and largely a pipe dream but moving nicely along now that the foundation on the programming level is stable). And for this part there is no comparison against Lua.

Why use LaTeX3 programming over Lua, when LuaTeX is available?

To build the upper layers it is extremely important to have a stable underlying base. As @egreg mentioned in chat: compare the package situation in 2.09 to the package situation in 2e. The moment there were standard protocols to build and interface packages the productivity increased a lot. However, the underlying programming level in 2e was and is still a mess which made a lot of things very complicated and often impossible to do in a reliable manner. Thus the need for a better underlying programming layer.

However, that programming layer is build on eTeX not because eTeX is the superior engine (it is compared to TeX but not with respect to other extensions, be it Lua or some other engine) but because it is a stable base available everywhere.

So eTeX + expl3 is a programming layer that the LaTeX3 team can rely on of not being further developed and changed (other than by us). It is also a base that is immediately available to everybody in the TeX world with the same level of functionality as all engines in use are implementing eTeX extensions.

Any larger level of modifications/improvements in the underlying engine is a hindrance to build the upper layers. True, some things may not work and some things may be more complicated to solve but the tasks we are looking at (well I am) the majority are very much independent of that layer anyway.

To make a few examples:

good algorithms for automatically doing complex page layout aren't there (as algorithms) so Lua will not help you here unless somebody comes up with such algorithms first.
something like "coffins", is thinking about approaching "design" and the importance here is how to think about it, not how to implement it (that comes second) -- see Is there no easier way to float objects and set margins? or LaTeX3 and pauper's coffins for examples

Having said that, the moment LuaTeX would be stable similar to eTeX (or a subset of LuaTeX at least) there might well good reasons for replacing the underlying engine and the program layer implementation. But it is not the focus (for now).

Why use LaTeX3 to program at all? Why not turn its development into the development of a LaTeX-style document design library for LuaTeX, written in Lua?

Could happen. But only if that "LuaTeX" would no longer be a moving target (because LaTeX3 on top would be moving target enough).

Side remark: @PatrickGundlach in his answer speculated that this answer that the LaTeX3 goal is backwards compatibility. Wrong. The same people that are coming down on you very strong about compatibility for LaTeX2e have a different mindset here. We do not believe that the interesting open questions that couldn't get resolved for 2e could be resolved in any form or shape with LaTeX3 in a document-compatible manner.

Input compatible: probably. But output compatibility for old documents, no chance if you want to get anything right.

But in any case, this is not an argument for or against implementing the ideas we are working on one day with a LuaTeX engine.

Is the separation of LuaTeX and LaTeX3 a result (or at least an artifact) of the non-communication among developers that Ahmed Musa described in his comment to this answer? What kind of cooperation is there between these two projects to reduce duplication of effort?

As I tried to explain, there is not much overlap in the first place. There is much more overlap in conceptual ideas on the level ConTeXt viz. LaTeX.

An even more fantastical notion is to implement every primitive, except \directlua itself, in terms of Lua and various internal typesetting parameters, thus completely divorcing the programming side of TeX from the typesetting side.

That brings us to a completely different level of discussion, namely is based on LuaTeX, or anything else for that matter, a completely different approach to a typesetting engine possible? That is a very interesting thought, but as @Patrick explained it isn't done with leaving TeX to do the typesetting and do everything else in a different language. So far such concepts have failed whether it was NTS or anything else because fundamentally (in my believe) we haven't yet grasped how to come up with a successful and different model for the TeX macro approach (as ugly as it might look in places).