[Tex/LaTex] Best Practices for Lua Modules

best practiceslualuatex

In the Lua world Lua Modules are installed using LuaRocks. With LuaTeX these modules are normally installed in the scripts folder (under MikTeX windows).

What tools are available for downloading and installing normal Lua packages? How can one avoid conflicts? For example in the minimal below the kpse.lookup returns the correct file path both with a directory prefix and without. Wouldn't this create a conflict, if two packages offer a Utilities module?

%!TEX TS-program = lualatex
\documentclass{article}
\usepackage{luatexbase}
\usepackage{luacode}
\begin{document}

\begin{luacode}
local path = "luaotfload/"
local filename = "mkstatus"
local src = path..filename
local filename = path.."mkstatus"
local srcdots = "luaotfload.mkstatus"
local f = kpse.lookup(filename, {format = "lua"})
local f1 = kpse.lookup(src, {format = "lua"})
-- success
tex.print("\\par",f)
tex.print("\\par",f1)

--  local mkstatus = require("mkstatus")

\end{luacode}

\end{document}

Is there a standard way for writing and installing modules for LuaTeX?

Best Answer

Lua modules have evolved and since Lua 5.2 you do not need the keyword module. Most probably you have seen modules written as mymodule.lua:

   module("mymodule", package.seeall)
        function foo() -- create it as if it's a global function
        print("Hello World!")
    end

Do not use this format as it is now depreciated.

Minimal module example

A Lua module is typically written so that it just returns a table. It is also saved in a file with the module's name. So if you have a module test, you should save it as test.lua

-- Module test (saved under test.lua)
local M = M or {}
M.test = function ()
      return "a test"
end
return M

Using the module

The module can be called by using the require() function. The convention here is to use a local variable with the same name as the module's name. If the module name has dots then use the name after the last dot.

local test = require"test"

The reason for the convention, is that it helps as a mnemonic for the module's name in long code, so you do not have to scroll up to see where it originated.

Submodules and packages

Lua allows module names to be hierarchical, using a dot to separate name levels. For instance a module named languages.greek is a submodule of the module languages. A package is a complete tree of modules and it is the unit of distribution in Lua.

From the point of view of Lua, submodules in the same package have no explicit relationship. Requiring a module does not automatically load any of its submodules; requiring languages.greek will load greek and not the languages and greek modules. It is good practice, if the package requires all the modules to write an initialization module that then loads the balance of the modules.

The folder structure should follow a russian dolls approach with the main module saved in a file with the modules name.

Where to develop modules

It is in my opinion a better practice to develop LuaTeX modules using an IDE such as ZeroBrane Studio or your favourite editor such as Vim or Emacs. The modules should be able to work independently of the LuaTeX engine, to enable proper testing. More about this later.

Documenting the Modules

While developing the modules for a package, my own experience leads me to believe that it is better to have them as self documented lua using luadoc or Ldoc, rather than heading straight into the ltxdoc\docstrip method. This enables the use of an IDE and the code development flows easier. Once you are satisfied with the code this can be transferred into a |.dtx| for easy distribution. This is easily generated as the rest of the package files using:

\nopreamble\nopostamble
\generate{
  \file{mymodule.lua}{\from{\jobname.dtx}{mymodule}}
}

A module besides its Lua comment strings might have build-in identification tables, for example:

-- module name
-- cldr based greek translation strings for territories
-- as defined by unicode

local m = m or {}

  m =  {
    el =  {
      identity =  {
        version =  {
          _cldrVersion = "26",
          _number = "$Revision: 10809 $",
          _version = '1.01',
          _author = "some name"
        },
        generation =  {
          _date = "$Date= 2014_08_14 15=10=07 _0500 (Thu, 14 Aug 2014) $"
        },
       },
  ...
  return m

One can also use luatexbase style for versioning and loading and registering the modules. This brings us back to paths, which is a pain when developing in Lua and LuaTeX.

Paths

More precisely, when LuaTeX asks for foo.bar (or foo.bar.lua), it first looks for file foo/bar using Kpathsea with the extension .lua, and then for foo.bar, removing the possible extension. So in order for a module to be found it needs to be in a directory where Kpathsea looks for Lua files. If you only have one file, you can locate it in the same folder as your |.tex| files. When you require a file that kpathsea cannot find the error message will point you to all the directories that were searched and this can give you a hint as to where you need to locate modules to have them accessible to LuaTeX.

Installing Lua packages (from Lua repositories such as Luarocks) will not automatically make them available to LuaTeX. So far the best method I have found, is to use git to install them. For example to install penlight, which has a set of useful utilities on your machine navigate to your \scripts directory in your distribution and type:

 git clone https://github.com/stevedonovan/Penlight.git

After the installation, you will have to update the distribution database.

Separation of concerns

Many utilities and modules, written for LuaTeX might have a utility outside the TeX world so it is good practice, in my opinion to split the pure Lua code in its own module and any code exporting to LuaTeX in a separate module. Lua talks to TeX via one of the tex.print and or similar for nodes and these should preferably be in different modules. Also it is good practice to also have some testing Lua code.

Here is a somewhat longer example to illustrate the discussion. Create a MWE as follows:

%!TEX TS-program = lualatex
\documentclass{article}
\usepackage{luatexbase}
\usepackage{luacode}
\begin{document}

\begin{luacode}
local f = kpse.lookup('cmd', {format = "lua"})
-- success
tex.print("\\par",f)
local cmd = require("cmd")
local csname, def = cmd.csname, cmd.def
csname('test', 'this is a test in csname')

def('test', 'this is a test')
\end{luacode}

\test
\makeatletter
\@test
\end{document}

This MWE will be our test of the module in LuaLaTeX. The interface Lua package we will create a moudle called cmd that it will create macros using \defs or csname. The code is below.

---
-- module cmd
-- 
-- require main  (not provided in example)
local m = m or {}
local print = print

if type(tex)== "table" then   
   print = tex.print
end

m.csname = function (a, b) 
  return print("\\expandafter\\gdef\\csname"..'@'..a.."\\endcsname{"..b.."}")
end  

m.def = function (a, b) 
  return print("\\gdef\\"..a.."{"..b.."}")
end  

return m

Note the use of print for both printing within a document or during pure Lua tests. We simply test if tex is available and redefine the print function.

Its Lua tests can be at a separate file or module (if we have a lot of tests). We name thsi file test_cmd

--- cmd_test

local cmd = require("cmd")

local csname, def = cmd.csname, cmd.def
csname('test', 'this is a test in csname')

def('test', 'this is a test')

In summary any module should have an associated module code for TeX and also for testing. A LuaTeX package should consist of

-Lua Modules (this we have not created in the example above)

-Associated Lua Modules interfacing with LuaTeX (as per example above)

-Testing modules (both for TeX as well as Lua)

The above has been my own personal experience so far, don't treat them as cast in stone. The LuaTeX Team has opened such a pandora's box and for the first time one can start thinking of proper well structured code. No amount of TeX hacking can ever replace complicated programming such as properly internationalizing a package or obtaining json responses from a website and then printing a table using Lua.

Related Question