[Tex/LaTex] Using LaTeX3’s xtemplate

Question

Relevant are the answers to Moving to LaTeX3 for package authors and What new bits have already been implemented in LaTeX3? Will my current documents (with many packages) still compile with LaTeX3?

How is xtemplate properly used? I’ve read the package documentation and Lars Hellström’s Some notes on templates, and I’ve tried to make sense of the xfrac implementation.

What I’ve gleaned is that there are “objects”, “templates”, “keys”, and “instances”. But how they interact is confusing, and xfrac has too many options to grasp easily (and is written assuming the reader understands xtemplate, rather than as an xtemplate tutorial).

Can the concepts of xtemplate be described in a simple-but-complete worked example?

Answer 1

Full example

In the first implementation of the template ideas there has been a full example that tried to work through the underlying ideas. On can still find the documentation of template.dtx on the web, e.g., here (in the directory /doc/latex/xpackages/xbase/). This is for a different implementation but the basic concepts haven't changed so it might be helpful still. Somehow during the reimplementation as xtemplate.dtxthis part of the documentation was dropped (right now that section is "empty").

The Concepts explained

The concepts implemented in xtemplate (which I consider these days as a second-level proto-type) are best explained using some pictures. Here is my rough sketch of what I think the LaTeX3 architecture should look like:

The xtemplate package builds out the concepts for the "object type repository", the "template repository", (parts of) the "Class functional specs", and the "Class layout specs". What it doesn't do, though, is interfacing with what is called "LaTeX Database (LDB)" because at the time the main work for xtemplate was done we thought that the concepts behind the LDB are not realistically implementable, so we put it aside and tried to implement a solution without it, which is largely what is the current xtemplate code.

Okay, so this doesn't tell you any better what this is all about, I guess. :-) So let's try to explain it from the middle part outwards. There we have the "Typesetting Element Layer" and the idea is that this describes all the "typesetting elements", i.e., objects that take inputs (0 or more) and do something with it to produce a "typeset result".

Typesetting Design Layer

The idea now is that these "typesetting elements" can be abstractly described by

• the inputs they take (and what they semantically mean)
• and the semantic of what they produce (for the sake of a better word)
• but not the visual representation

This is what we call the ObjectType and they end up in the "object type repository". As an example think of a "heading" object type that takes a number of inputs, e.g., the heading title, the TOC title, whether or not it is numbered, etc. (some of those inputs could be special values like "NoValue") but for sake of speed etc the current implementation assumes each type has always a fixed number of arguments and they are positional.

Now so far this says nothing about how things should be visualized. It just gives abstract functional elements.

For each ObjectType there are a number of templates (one or more) in the "template repository" that all implement a visualization of the element, ie, they do the typesetting. They are called templates because they are intended to offer a certain flexibility in providing design choices. Whether designs are split over different templates or are implemented in a very complex template is a bit of a matter of taste and practicality, e.g., one may have a template for generating display headings and one to generate run-in headings or attempt to have a more complex template that can produce both layouts (like LaTeX2e's \@startsection tries).

xtemplate provides a command (\DeclareTemplateInterface) to declare a template interface which is basically the description of the knobs and wistles the template offers to manipulate the design it implements. This is given as a list of keys which can have a number of input types. This is, if you like the designers side of the house: to instantiate a template the designer would need to give such keys values.

It also provides a declaration for the template code (\DeclareTemplateCode) that implements the design taking the interface key values as input and the document arguments from the ObjectType. This coding would be done using the expl3 layer (i.e., the "CoreLanguage Layer") with the help additional help of what I termed the "Typesetting foundation layer" (of which not much exists so far).

All this is in the box "template repository".

Now if none of the available templates implement the design the designer is looking for, then another template has to be coded that implements the desired design for the ObjectType first, otherwise the task of a designer is to select a suitable template and initiates it with some values to achieve the wanted result.

Class Design Layer

Moving up the stack we have two bricks in the "Class Design Layer": The "Class functional specs" and the "Class layout specs". Note that this layer doesn't deal with the document level syntax. We are still on the level of (named) ObjectTypes and those take inputs in a standarized form.

Class functional spec

The "Class function spec" describes what are the logical elements are that a document class provides, e.g., that it provides the following heading levels: "A-head", "B-head", down to "F-head" say, or in 2e terms "section", "subsection", etc, down to "subparagraph".

It should also describe the relationships (not really there (yet)) so in essence the idea of this is to provide a kind of "class DTD".

The idea is that if a document belongs to a certain document class (as described by this functional spec) its visualization could be changed from one form to another by replacing on layout spec conforming to the functional spec with another conforming to this functional spec.

In other words a certain functional spec should be common for a number document class layouts, which would make those layouts interchangeable without otherwise touching the document. As we know this is partially possible in 2e but often not quite, e.g. article.cls can be replaced by amsart.cls but some things will fail because the DTDs are not fully the same unfortunately (and in 2e there is no separation of the DTD from the visualisation both is done in the same place, i.e. in the .cls file so it is not that surprising).

Technically speaking this block is not properly covered by xtemplate at the moment it only exists conceptually, see discussion of deficiencies below. The closest you get here is \UseInstance but this is not a declaration that can stand on its own, but something to be used one layer up.

Class layout spec

The other building block handles the real instantiation for a specific class, e.g., if the functional spec says there is an "A-head" and a "B-head" then here we would have

 \DeclareTemplateInstance{heading}{A-Head}{some-template}
     {key1=val1, key2=val2 ...}
 \DeclareTemplateInstance{heading}{B-Head}{maybe-some-other-template}
     {key1=val1, key2=val2 ...}

So both together describe what a specific document class provides in terms of elements and how these elements are formatted.

The only thing that is still missing is how such elements are encoded in real documents, i.e. what syntax is used to input them. That is subject to the next layer up.

User Interface Layer

The top-level layer turns document input syntax into the format used on the "Class Design Layer (functional spec)". An example of this would be the xparse implementation that provides parsing functionalities a la LaTeX2e with some extra bells and whistles. But instead of this one could provide a different user syntax, e.g., some xml interface or ...

Putting the stuff together, using the current possibilities we would then have something like

\DeclareDocumentCommand \section { * o m }   % we implement 2e's interface
   { \UseInstance{heading}{A-head} {#1} {#2} {#2} {#3} }

If you wonder why this looks so funny: For the sake of an argument :-) the above assumes that the ObjectType "heading" was defined to require 4 arguments:

• a boolean value (indicating if the heading should not get numbers
• a text for the TOC or "NoValue"
• a text for the running heading or "NoValue"
• a text for the heading title

As the 2e document level interface has only one optional argument we duplicated it for both TOC and running-head (as it is also done in 2e) Instead we could have offered a different top-level syntax like

\DeclareDocumentCommand \section { * o m o } 
   { \UseInstance{heading}{A-head} {#1} {#2} {#4} {#3} }

that does make use of all input arguments the object type "heading" has (only in a different order)

or ...

Summary

To summarize here is a slightly different view on the architecture just described (ignore the LDB part for now):

And here yet another one looking at it more from the perspective of the different roles and how things are be specified using the declaration possibilities of xtemplate:

You can see four different roles:

• the Document Type Designer defines what kind elements are in a document class (and possibly defines ObjectTypes if they are not yet in the object repository). He is also listed as the one who defines the template interface, but I guess I marked that wrong -- that should probably be the designer doing that.
• if a template actually needs to be produced that would be the task of a Programmer who really understands how to program in expl3 and all the goodies :-) but hopefully for most requirements usable templates exist (after a while)
• the of the Designer is then to select suitable templates and decided on the right kind of values for their keys, and if no suitable template exist, ask a Programmer to build one.
• and finally there is the Author whose task is to concentrate on writing a real document and not worrying about all the other stuff

Of course, in real life those roles might end up being played by a single person.

The deficiencies

... or what is wrong in my opinion with the current proto-type. Well, what the current implementation provides is an approach to static design:

• here is a abstract object (type), say a heading that takes a defined set of inputs
• here is a bunch of template that typeset such an object
• so to make a design, select a template and select the right parameters and you are done

Simple, right? Well not quite, because often document design needs flexibility depending on context. Take for example display headings:

• they have a defined space above the heading (say preskip)
• they have a possibly different space below the heading (postskip)

But what happens if one heading directly follows another one as in

\section{A-head}
\subsection{B-head}

What is the space between them?

• the sum postskip(A-head)and preskip(B-head)?
• the maximum of both?
• the minimum?
• something else?

LaTeX2e solves this trivially by using \addvspace which is implementing the second solution, i.e., the maximum. But this is fairly limiting the design possibilities.

One can think of other possibilities, e.g. solving this somewhat adhoc so that a heading can detect that it is directly following another heading and then using a different key, e.g. between-heading-skip. But this is conceptually rubbish because

• all decisions would be placed into the second heading and not really on the knowledge that it is an A-head followed by a B-head
• it would suddenly require the templates to understand and manage context information which makes them complex and if a single template is not obeying to some special protocol all the identification, etc. would break down like a house of cards.

So this is an example "context" because some elements are in sequence. Another example is that elements behave quite differently because they are in different areas of the document. For example

• headings are differently numbered in frontmatter, mainmatter and backmatter
• lists like itemize need totally different spacing if used in contexts with different font sizes, or in special regions (e.g., you may want a totally different design for them if such a list appears in a boxed area in the margin.

Basically none of this is supported properly by xtemplate.

Move the film back: in the early '90s (yes it is that long ago) we had grand ideas with something that we called LDB back then that would allow to manage context dependencies of high complexity. To some extend it was a pre-CSS (not quite but CSS came up with some of those ideas a couple of years later) and that would have fairly perfectly managed all this complexity. However back then these ideas were simply years too early and we finally came to the conclusion that it is just technically not working.

Times have changed: the problem is still not solved but computers got so much faster that I'm now fairly determined to retry this, see my talk given at TUG 2011 in India [YouTube] (not surprisingly you will see some of the above pictures there in the slides).

So back then we implemented the LDB (in two competing approaches but both were too slow and took too much memory, so be it onto the shelf and instead we came up with the "simpler" approach in template.dtx in 199x which then later was reimplemented by Joseph to become xtemplate.dtx. And for the "static" design part the template approach is really good and getting you somewhere. But the dynamic part remained and so we tried to solve this in a simpler fashion.

One idea we had was to provide "collection instances", basically that you can define several "collections" of instances and then switch between them. But this is such a crippled and restricted way of describing context that I'm totally convinced now this is a dead end. So consider this as the "rubbish part" in the concept which is also why I didn't describe it above (but it is still documented in xtemplate even with some warning, so be aware).

Instead what I think needs happening is to bring together the ideas from template and from LDB for context and offer them in a coherent way --- and nowadays this is possible and I believe I have a good vision on how this should look like.