[Tex/LaTex] Why does running LuaLaTeX from the command line not produce the same aux files as building from Sublime Text

luatexsublime-text

I'm trying to automate a LaTeX build and one step in my workflow is invoking LuaLaTeX from the command line like so:

lualatex example.tex

Although running this command outputs a PDF as required, it is missing an image which needs to appear on the upper right hand corner.

That being said, pressing Ctrl + B (build) in Sublime Text (after setting the builder to LuaLaTeX) produces the PDF with the desired picture in the correct position. I'm wondering what exactly could be happening behind the scenes here for this to work. Clearly just typing lualatex example.tex is insufficient, so what am I missing?

From what I can see,invoking the Sublime Text LuaLaTeX builder produces a directory called aux/ in the working directory. Inside aux/ is a number of auxiliary files:

example.aux
example.log
example.fdb_latexmk
example.fls
example.synctex.gz
example.pdf

However, simply running lualatex example.tex from the command line only generates the first two files in that list, and I suspect this might be part of the reason my picture isn't showing up in the PDF.

The method by which the picture gets placed into the document is a bit complicated. The file path to the image is stored in a JSON file called parameters.json. This path is imported into my .tex file like so:

\begin{document}
    \begin{luacode}
        require("lualibs.lua")

        function getjsonfile (file)
            local f, s
            f = io.open(file, 'r')
            s = f:read('*a')
            f.close()
            return s
         end

        paramtable = utilities.json.tolua(getjsonfile('parameters.json'))
    \end{luacode}

    \Header{\luadirect{
        local path = paramtable["image_path"]
        tex.print(path)}}

\end{document}

Here, \Header is a macro that accepts either a valid file path to an image or the string none, in which case no image is displayed.

Since the key image_path points to a file (extension is ignored), \Header{} receives its argument and everything should be fine in theory.

I'm quite new to LaTeX/LuaLaTeX so I would appreciate any help in figuring out what exactly is the difference between building from Sublime Text and running the command right from the command line. I've included a minimum working example here, along with a copy of my LaTeXTools.sublime-settings file, which contains information about my build.

MWE – parameters.json:

{"image_path": "./picture"}

MWE – example.tex:

\documentclass[paper=a4,fontsize=11pt]{scrartcl} % KOMA-article class
\usepackage{luacode}
\usepackage[left=1.5cm, right=1.5cm, top=1.5cm, bottom=1.5cm]{geometry}
\usepackage{graphicx}                    % Enable pdflatex
\usepackage{etoolbox}
\usepackage{tikz}
    \usetikzlibrary{calc}   

\newcommand{\placepicture}[4][center]{%
  % [#1]: box anchor: center (default) | 
  %                 south west | west | north west | north |
  %                 north east | east | south east | south | 
  %                 mid west | mid | mid east |
  %                 base west | base | base east 
  % #2: horizontal position (fraction of page width)
  % #3: vertical position (fraction of page height)
  % #4: image name
  %
  \tikz[remember picture,overlay,x=\paperwidth,y=\paperheight]{%
    \node[anchor=#1,inner sep=0pt]
    at ($(current page.south west)+(#2,#3)$) {\includegraphics[width=0.2\textwidth]{#4}};
  }%
}

\newcommand{\Header}[1]{
    \edef\mytemp{{#1}}%
    \expandafter\ifstrequal\mytemp{none}{}{\placepicture[north east]{0.9}{0.98}{#1}}
}

%%% ------------------------------------------------------------
\begin{document}
    \begin{luacode}
        require("lualibs.lua")

        function getjsonfile (file)
            local f, s
            f = io.open(file, 'r')
            s = f:read('*a')
            f.close()
            return s
         end

        paramtable =  utilities.json.tolua(getjsonfile('parameters.json'))
    \end{luacode}

    \Header{\luadirect{
            local path = paramtable["image_path"]
            tex.print(path)}}

  Some normal body text here.

\end{document}

LaTeXTools.sublime-settings (not the whole file, just build settings):



{
// ------------------------------------------------------------------
// Temporary file settings
// ------------------------------------------------------------------
    // Ends of the names of temporary files to be deleted
    "temp_files_exts": [
        ".blg",".bbl",".aux",".log",".brf",".nlo",".out",".dvi",".ps",
        ".lof",".toc",".fls",".fdb_latexmk",".pdfsync",".synctex.gz",
        ".ind",".ilg",".idx"
    ],
    // Folders that are not traversed when deleting temp files
    "temp_files_ignored_folders": [
        ".git", ".svn", ".hg"
    ],

// ------------------------------------------------------------------
// Platform settings: adapt as needed for your machine
// ------------------------------------------------------------------

    "linux" : {
        // Path used when invoking tex & friends; MUST include $PATH
        "texpath" : "$PATH:/usr/texbin",
        // Command to invoke Python. Useful if you have Python installed in a
        // non-standard location or want to use a particular version of python.
        // Both Python2 and Python3 are supported, but must have the DBus bindings
        // installed.
        "python": "",
        // The name of the ST2 or ST3 executable. On Ubuntu, both subl and sublime-text are
        // available for ST2; adjust as needed for other platforms, and for ST3
        "sublime": "sublime-text",
        // How long to wait after evince or okular has launched before sending a sync message
        // in seconds, floating point; choose 2.0 or 3.0 on a slower machine, 0.5 on a fast one
        // Note: only tweak this if sync after launching the PDF viewer does not seem to work,
        // or if the PDF viewer opens instantly and you don't want to wait.
        // Default: 1.5 (works on my MBP4,1...)
        "sync_wait": 1.5,
        // Command to invoke Sublime Text. Used if the keep_focus toggle is true.
        // If blank, "subl" or "sublime_text" will be used.
        "sublime_executable": "",
        // how long (in ms) to wait after the jump_to_pdf command completes
        // before switching focus back to Sublime Text. This may need to be
        // adjusted depending on your machine and configuration.
        "keep_focus_delay": 0.5
    },

// ------------------------------------------------------------------
// Output Directory settings settings
// ------------------------------------------------------------------

    // OPTION: "aux_directory"
    // Specifies the auxiliary directory
    // Possible values:
    //
    // ""               the default; does not use any auxiliary directory
    //
    // path             the path to the auxiliary directory; if this is
    //                  not an absolute path it is interpreted as a
    //                  path relative to the main tex file
    //
    // "<<temp>>"       the auxiliary directory will be a temporary
    //                  directory generated in as secure a manner as
    //                  possible; note that this temporary directory
    //                  is only valid until ST is restarted and will
    //                  be deleted on the next start-up
    //
    // "<<project>>"    this creates an auxiliary directory in the same
    //                  folder as the main tex file; the name is the
    //                  MD5 hash of the absolute path of the main file;
    //                  unlike <<temp>> this directory will persist
    //
    // "<<cache>>"      this creates an auxiliary directory in the ST
    //                  cache directory on ST3 or a suitable directory
    //                  on ST2; unlike <<temp>> this directory will
    //                  persist; unlike <<project>>, it will not be
    //                  in the same directory as the main tex file
    //
    // NOTE: This setting will be overridden by the corresponding
    // %!TEX directive if any; also, while it is possible to define
    // a global value here, it may make more sense to define this
    // value in your project settings if you use STs project feature
    // if you do so, however, note that the path will be interpreted
    // relative to the location of your project file

    "aux_directory": "",

    // OPTION: "output_directory"
    // Specifies the output directory

    "output_directory": "./aux",

    // OPTION: "jobname"
    // Specifies the jobname to use when building the document

    "jobname": "",

    // OPTION: "copy_output_on_build"
    // Specifies whether to copy the final PDF file to the same folder
    // as the main tex file; if it is neither true nor false it must be
    // as list of extensions of the files to copy into the same folder as
    // the main tex file; this  only applies if an output directory is
    // set via a setting or a `%!TEX` directive

    "copy_output_on_build": true,

// ------------------------------------------------------------------
// Build engine settings
// ------------------------------------------------------------------

    // OPTION: "builder"
    // Specifies a build engine
    // Possible values:
    //
    // "default" or ""  the default built-in build engine; currently
    //                  this is the same as "traditional"
    //
    // "basic"          invokes pdflatex / xelatex / lualatex as
    //                  needed, then biber / bibtex and pdflatex /
    //                  xelatex / lualatex again if needed. Unlike
    //                  the "simple" builder this supports most of
    //                  LaTeXTools builder features.
    //
    // "script"         external script: invokes the set of commands
    //                  specified in the "script_commands" setting
    //                  in the platform-specific part of the
    //                  "builder_settings"
    //
    // "simple"         invokes pdflatex 1x or 2x as needed, then
    //                  bibtex and pdflatex again if needed;
    //                  intended mainly as a simple example for
    //                  people writing their own build engines.
    //
    // "traditional"    replicates the 'old' system based on
    //                  latexmk (TeXLive) / texify (MiKTeX)
    //
    // custom name      you can also use third-party build engines;
    //                  if so, set the "builder_path" option below
    //
    // NOTE: custom builders CANNOT have the same name as an existing
    // built-in build engine (including "default")

    "builder": "basic",

    // OPTION: "builder_path"
    // If non-empty, specifies a path to a custom builder, relative to the
    // Sublime Text Packages directory.
    // For instance, "User/builders" (on Windows: "User\builders") is a good
    // choice if you roll your own.
    // (Note: if you choose "User", you may get a Python import error in the
    // console, but things will still work).
    // Leave empty ("") for a built-in builder.

    "builder_path": "",

    // OPTION: "builder_settings"
    // Specify builder-dependent settings and preferences
    // Possible values: see README or documentation provided with
    // third-party build engine.
    // Builder setting can be general or OS-dependent

    "builder_settings" : {

        // General settings:
        // See README or third-party documentation

        // (built-ins): true shows the log of each command in the output panel
        "display_log" : false,
        // Enable this when using the minted package 
        "options": ["--shell-escape"],

        // Platform-specific settings:
        "osx" : {
            // See README or third-party documentation
        },

        "windows" : {
            // See README or third-party documentation
        },

        "linux" : {
            // See README or third-party documentation
        }
    },

// ------------------------------------------------------------------
// Build panel and phantoms settings
// ------------------------------------------------------------------
    // OPTION: "highlight_build_panel"
    // adds syntax highlighting to the build panel so that errors, warnings, etc.
    // show up in different colors; if set to false, the panel will be left as-is.
    "highlight_build_panel": true,


    // OPTION: "hide_build_panel"
    // level to hide the build panel after the build is finished
    // Possible values are:
    // "always" (hide the panel even if the build failed),
    // "no_errors" (only hide the panel if the build was successful even with warnings),
    // "no_warnings" (only hide the panel if no warnings occur)
    // "no_badboxes" (only hide the panel if no badbox messages occur when badboxes are enabled) and
    // "never" (default, never hide the build panel)
    "hide_build_panel": "no_badboxes",

    // OPTION: "display_bad_boxes"
    // controls whether or not to display any bad boxes in the build output
    // if this is not set to true, the setting "no_badboxes" for
    // "hide_build_panel" is equivalent to "no_warnings"
    "display_bad_boxes": false,

    // OPTION: "show_error_phantoms"
    // (ST3, Build 3118+ only)
    // level to show error phantoms in the file
    // Possible values are:
    // "none" (don't show any phantoms at all)
    // "errors" (only show errors, which breaks the compilation)
    // "warnings" (default, show errors and warnings)
    // "badboxes" (show errors, warnings, and badboxes)
    "show_error_phantoms": "warnings",

    // number of seconds to display the "build succeeded" or "build failed"
    // message
    "build_finished_message_length": 2.0,

// ------------------------------------------------------------------
// Viewer settings
// ------------------------------------------------------------------
    // OPTION: "viewer"
    // Specifies which viewer to use
    // Possible values:
    //
    // "default" or ""  the default viewer for your platform, which
    //                  is sumatra on Windows, skim on OS X and
    //                  evince on linux
    //
    // "command"        invokes a viewer based on commands configured
    //                  in viewer settings below; see the README
    //                  for details
    //
    // "evince"         uses Evince; this is the default on linux
    //
    // "okular"         uses Okular to open the pdf
    //
    // "preview"        uses Preview.app to open the pdf
    //
    // "skim"           uses Skim to open the pdf; this is the
    //                  default on OSX
    //
    // "sumatra"        uses SumatraPDF; this is the default on
    //                  Windows
    //
    // "zathura"        uses Zathura to open the pdf
    "viewer": "",

    // OPTION: "viewer_settings"
    // Specify viewer-dependent settings and preferences
    // Possible values: see README or documentation provided with
    // third-party viewer.
    // Viewer settings can be general or OS-dependent
    "viewer_settings": {
        // Platform-specific settings:
        "osx" : {
            // See README or third-party documentation
        },

        "windows" : {
            // See README or third-party documentation
        },

        "linux" : {
            // See README or third-party documentation
        }
    },

    // OPTION: "open_pdf_on_build"
    // specifies whether LaTeXTools should open the PDF file on a
    // successful build. If set to false, the PDF file won't be opened
    // unless explicitly launched using C-l,v or C-l,j
    "open_pdf_on_build": true,

    // OPTION: "disable_focus_hack"
    // if set to true, this will stop LaTeXTools from attempting to steal focus
    // from the viewer. NOTE: This does not mean that the *viewer* won't steal
    // the focus, only that LaTeXTools won't try to steal the focus back.
    "disable_focus_hack": false,

// ------------------------------------------------------------------
// Opening files included into the tex source code
// ------------------------------------------------------------------

    // image types you use in latex
    // these types will be used for autocompletion and
    // opening of included images, when no extension is written
    "image_types": ["png", "pdf", "jpg", "jpeg", "eps"],


// ------------------------------------------------------------------
// Cache options
// ------------------------------------------------------------------

    // settings for caches to update on load
    // leaving these as `true` will ensure LaTeXTools pre-caches the appropriate
    // data when a TeX document is loaded; setting these to `false` will
    // cause the cache to be built when first needed
    // "cache_on_load": {
    //  // analysis: the internal view that LaTeXTools has of your document
    //  "analysis": true,
    //  // bibliography: ensures the bibliography is parsed and cached
    //  "bibliography": true
    // },

    // settings to update caches when a document is saved
    // leaving these as `true` will ensure LaTeXTools reloads the data on save,
    // if necessary; setting these to `false` will cause the cache to be
    // re-built according to its rules
    // "cache_on_save": {
    //  // analysis: the internal view that LaTeXTools has of your document
    //  "analysis": true,
    //  // bibliography: ensures the bibliography is parsed and cached
    //  "bibliography": false
    // },

    /* The life-span of the local cache.
    After this life-span the local cache will automatically be invalidated and refreshed.
    You can invalidate the cache manually by removing all temporary files `C-l,backspace`.
    If the value is smaller then the functionalities are more up-to-date,
    but more recalculations might decrease the performance.

    The format is "X d X h X m X s", where X is a natural number
    "s" stands for seconds, "m" for minutes, "h" for hours, and "d" for days.
    Missing fields will be treated as 0 and white-spaces are optional.
    Hence you can write "1 h 30 m" to refresh the cached data every one and a half hours.
    If the string is invalid the default value (30 minutes) will be used.
    If you use `infinite` the cache will not invalidated automatically.
    */
    "local_cache_life_span": "30 m"
}

I've included the diffs of the LaTeX logs of both of my attempts:

  • The one where the image was successfully displayed.
  • The one where no image was displayed.

Log diffs:

1,2c1,2
< This is LuaTeX, Version 1.0.4 (TeX Live 2017)  (format=lualatex 2017.11.12)  16 OCT 2019 22:54
<  system commands enabled.
---
> This is LuaTeX, Version 1.0.4 (TeX Live 2017)  (format=lualatex 2017.11.12)  16 OCT 2019 22:55
>  restricted system commands enabled.
37c37
< luaotfload | main : initialization completed in 0.256 seconds
---
> luaotfload | main : initialization completed in 0.263 seconds
817,818c817
< (./aux/example.aux)
< \openout1 = example.aux
---
> No file example.aux.
819a819
> \openout1 = example.aux
834d833
< 
955c954,958
< <./picture.jpg>] (./aux/example.aux))
---
> <./picture.jpg>] (./example.aux)
> 
> LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.
> 
> )
958c961
<  18576 strings out of 494483
---
>  18571 strings out of 494483
967c970
< Output written on example.pdf (1 page, 53837 bytes).
---
> Output written on example.pdf (1 page, 53834 bytes).

The diffs show an error: No file example.aux, but this is strange because that file is being generated.

Please let me know if I can supply any additional information.

Best Answer

The interesting bit of your diff is the following part

817,818c817
< (./aux/example.aux)
< \openout1 = example.aux
---
> No file example.aux.
819a819
> \openout1 = example.aux
834d833
< 

as well as the

> LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.

in the manual run.

This shows that in the Sublime-latexmk run (Lua)LaTeX gets to read an .aux file, but in your manually started lualatex run there is no .aux file.

The .aux file is a temporary auxiliary file that LaTeX uses to communicate with itself because it can't look into the future. The .aux file captures relevant information about your document (page references, label definitions, ...) that might be needed before they are issued in the document flow. It is read at the beginning of the document so that the information is present right from the start when needed. The file is written during each LaTeX run (unless explicitly requested otherwise). Some more background why this is needed can be seen in the \label example at Understanding how references and labels work.

This tells us that the .log of your Sublime-latexmk run is probably the .log file of a subsequent LuaLaTeX run on your document, while your manual lualatex run was a first run on your document (or the temporary files were deleted).

Indeed latexmk was specifically written with the aim of automatically running LaTeX and helper programs such as BibTeX, makeindex as often as necessary for the document to come out properly. If you run lualatex manually, you have to take care of doing that yourself.

In this case running lualatex on your document twice may already be enough (you may want to run in three times, just to be sure). In any case you will want to be sure that your .log file does not say

LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.

any more.

Run

lualatex example
lualatex example

Alternatively, you can use latexmk from the command line if you like

latexmk -lualatex example
Related Question