[Tex/LaTex] How to build a Makefile that builds a LaTeX document with dependencies in one run

latexmkluatexmakefile

I am trying to create a Makefile for my paper where I can type just make and it will compile everything in one go. The paper has several figures which are generated from scripts. However, everything I've tried requires several runs of make to get both the scripts to run and the paper itself.

I started with something based on this answer

all: paper.pdf

paper.pdf:

%.pdf: %.tex
    $(LATEXMK) -lualatex -halt-on-error -pdf -M -MP -MF $*.d $*

some-plot.pgf:
    script_to_generate_plot.py

-include *.d

However, from a clean build, or whenever I add a plot, this requires running make twice, once to generate the .d file and once to actually build the paper. And annoyingly the first time you just have to bypass an annoying "file not found" prompt.

I then noticed that latexmk has a flag to use make to build missing files, so I'm trying that:

all: paper.pdf

paper.pdf:

%.pdf: %.tex
    $(LATEXMK) -lualatex -interaction=nonstopmode -pdf -dvi- -ps- -use-make $*

some-plot.pgf:
    python script_to_generate_plot.py

Now what this does is finds one plot missing, build it, then fails with

Latexmk: 'pdflatex': source file 'some-plot.pgf' doesn't exist. I'll try making it...
------------
Running 'make "some-plot.pgf"'
------------
python script_to_generate_plot.py
Latexmk: Summary of warnings:
  Latex failed to resolve 1 reference(s)
Latexmk: Errors, so I did not complete making targets
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 256
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.
make: *** [paper.pdf] Error 12

If I run make again that plot was built, but it does the same thing with another one.

If I keep running make it eventually builds all my plots one by one (I have several plots, some of which take a while to generate so I'd like to keep the scripts to generate them separate).

How can I make it so that it builds everything in one go, regardless of which and how many files need to be regenerated?

EDIT I guess I forgot to mention that my other goal here is to avoid manually stating every .tex/.pgf/.pdf file dependency in my Makefile. If I do that, there's no point to using latexmk. I've done the manual Makefile before, but I'm always forgetting to update the dependency graph whenever I add a new file or rearrange things. My understanding is that the strength of latexmk is that it can figure that stuff out automatically. If I can get something that does that, that would be ideal. That and latexmk automatically rebuilds the paper and bib files as many times as necessary.

EDIT 2
I tried using the suggestion from the output, to use the -f flag to latexmk. This builds about 5 or so plots, then errors with

Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'origen-meeseeks.pgf'
Latexmk: Maximum runs of pdflatex reached without getting stable files
Failure to make 'paper.pdf'
Latexmk: Errors, in force_mode: so I tried finishing targets
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 256
Latexmk: Did not finish processing file 'paper':
   'pdflatex' needed too many passes
make: *** [paper.pdf] Error 12
transmutagen-papermaster*=$latexmk -

So it almost does what I want. How do I increase the max number of passes?

Best Answer

As I noted in my edit, if you add the -f flag to latexmk, it continues after each plot is generated and tries again.

Unfortunately, these "re-runs" are counted as part of the number of times latexmk calls latex to build the document. By default, it only does 5 of these, but you can modify this with a configuration parameter. Create a file .latexmkrc in the directory with your tex files and add

$max_repeat = 10

(replace 10 with some number bigger than the number of plots you have, probably at least 5 more than that).

I personally would consider these to be bugs in latexmk (building more than a single thing with make shouldn't require -f, each call to make shouldn't count against the max_repeat).

You also do need to add the file dependencies manually in the Makefile

paper.pdf: some-plot.pgf

I don't know if there's a way to make the -MF stuff from latexmk work properly with just one Make run that both generates and uses them.

Related Solutions

[Tex/LaTex] Use MiKTeX option through latexmk

I've made a new version of latexmk, which supports -aux-directory and -output-directory. It's v. 4.27a and can be found at

http://www.phys.psu.edu/~collins/latexmk/versions.html

This version ~~will be submitted to CTAN fairly soon, but some feedback would be useful, since I have not tested the new features extensively, especially as I don't have current access to MiKTeX~~ or a more recent one can be found on CTAN as well.

Latexmk now has options -auxdir, -aux-directory, -outdir, and -out-directory, plus corresponding configuration variables $aux_dir and $out_dir. See the documentation for more details. To achieve what the original poster wanted, it's sufficient to put

$aux_dir = 'C:/Users/doncherry/Documents/LaTeX/AUXI_global';

in a latexmkrc file. (I've used forward slashes in directory names since these are acceptable to MS-Windows, and avoid running afoul of quoting conventions.)

I've done some tests with the -output-directory option, and the only problem that needs sorting out is that the combination of biblatex with bibtex doesn't work because a file needed by bibtex isn't in its usual place. (Biblatex with biber works, as do ordinary uses of bibtex.) If you don't use biblatex, the last issue doesn't affect you.

[Tex/LaTex] Makefile that prevents tex directory from being cluttered with build output

I got it, maybe this helps someone else or it serves as an inspiration for creating Makefiles. The problem was the cd command and how make executes commands, like i mentioned i a comment above.

Makefile:

#
# Makefile for Bachelorthesis
#
#   Andreas Linz -- 10INB-T
#   HTWK-Leipzig, Fakultaet IMN
#   klingt.net@googlemail.com
#
# @ supresses command output
#
# conditionals are not allowed to have a starting tab, 
# otherwise they will be sent to the shell
#
DEBUG='true'
ROOT_DIR=$(shell pwd)
BUILD_DIR=$(ROOT_DIR)/.build
OUTPUT_DIR=$(ROOT_DIR)/output
#.PHONY : clearscr clean screen print

all: copy_sources screen print copy_output

copy_sources:
    @echo "Copying sources to build folder: $(BUILD_DIR)"
    @rsync --verbose --checksum --recursive --human-readable --progress --exclude=.build --exclude=.build_scripts --exclude=output --exclude=.git --exclude=.gitignore $(ROOT_DIR)/ $(ROOT_DIR)/.build

copy_output:
    @echo "Copying generated PDFs to output folder: $(OUTPUT_DIR)"
    @cp -f $(BUILD_DIR)/*.pdf $(OUTPUT_DIR)

screen:
    @echo "Building screen version ..."
ifeq ($(DEBUG) , 'true')
    .build_scripts/screen.sh $(BUILD_DIR) $(DEBUG) 
else
    .build_scripts/screen.sh $(BUILD_DIR) $(DEBUG)
endif

print:
    @echo "Building print version ..."
ifeq ($(DEBUG) , 'true')
    .build_scripts/print.sh $(BUILD_DIR) $(DEBUG)       
else
    .build_scripts/print.sh $(BUILD_DIR) $(DEBUG)
endif

clean_conflicts:
    @find $(ROOT_DIR) -name '*in Konflikt stehende*' -delete
    @find $(ROOT_DIR) -name '*conflicted copy*' -delete

clean: clean_conflicts
    @echo "I will clean up this mess ..."
    @cd $(BUILD_DIR); rm -r *

clean_all: clean
    @rm $(OUTPUT_DIR)/*

clearscreen:
    clear

# remove the following commands, if the build was started in the BUILD_DIR successfully
#   @rm -f *.out *.nav *.snm *.toc *.log *.bcf *.bbl *.blg *.lof *.lol *.lot *.run.xml *.xdy *.glo *.glg *.gls *.idx *.ist *-blx.bib .*.bbl.swp *.mtc* *.maf
#   @find . -name \*.aux  -type f -delete
#   @find . -name \*.bbl  -type f -delete
#   @find . -name \*.bak  -type f -delete
#   @find . -name \*.blg  -type f -delete
#   @find . -name \*.fls  -type f -delete
#   @find . -name \*.fdb_latexmk  -type f -delete
#   @find . -name \*.vsdx  -type f -delete
#   @find $(ROOT_DIR) -name '*in Konflikt stehende*' -delete

Build-Scripts:

screen.sh:

#!/bin/bash
#
# script for generating the print pdfs
# this script should be called from the Makefile in the document root
#
# check number of arguments with `$# -eq 0`
#

if [ $# == 2 ]; then
    echo "Changing directory to: $1"
    cd $1
    if [ $2 = 'true' ]; then
        echo "12345: $XELATEX_OPTS"
        echo -e "\n------------------ xelatex --------------------\n"     
        xelatex $XELATEX_OPTS thesis #|grep --ignore-case --extended-regex "info|warning|error|^\([A-Za-z0-9]*\)"
        echo -e "\n------------------- biber ---------------------\n" 
        biber $BIBER_OPTS thesis
        echo -e "\n--------------- makeglossaries ----------------\n"         
        makeglossaries thesis
        echo -e "\n------------------ xelatex --------------------\n" 
        xelatex $XELATEX_OPTS thesis > /dev/null
    else
        xelatex $XELATEX_OPTS thesis > /dev/null  
        biber $BIBER_OPTS thesis > /dev/null          
        makeglossaries thesis > /dev/null   
        xelatex $XELATEX_OPTS thesis > /dev/null
    fi
else
    echo "Number of arguments given: $#"
    echo "Usage: ./screen.sh path_to_build_dir [true/false]"
fi

print.sh:

#!/bin/bash
#
# script for generating the print pdfs
# this script should be called from the Makefile in the document root
#
# check number of arguments with `$# -eq 0`
#

XELATEX_OPTS=""
BIBER_OPTS=""

if [ $# == 2 ]; then
    echo "Changing directory to: $1"
    cd $1
    if [ $2 = 'true' ]; then
        echo -e "\n------------------ xelatex --------------------\n"     
        xelatex $XELATEX_OPTS thesis_print > /dev/null
        echo -e "\n------------------- biber ---------------------\n" 
        biber $BIBER_OPTS thesis_print 
        echo -e "\n--------------- makeglossaries ----------------\n"         
        makeglossaries thesis_print
        echo -e "\n------------------ xelatex --------------------\n" 
        xelatex $XELATEX_OPTS thesis_print | grep --ignore-case --extended-regex "info|warning|error|^\([A-Za-z0-9]*\)"
    else
        xelatex $XELATEX_OPTS thesis_print > /dev/null  
        biber $BIBER_OPTS thesis_print > /dev/null          
        makeglossaries thesis_print > /dev/null   
        xelatex $XELATEX_OPTS thesis_print > /dev/null
    fi
else
    echo "Number of arguments given: $#"
    echo "Usage: ./screen.sh path_to_build_dir [true/false]"
fi

Best Answer

Related Solutions

[Tex/LaTex] Use MiKTeX option through latexmk

[Tex/LaTex] Makefile that prevents tex directory from being cluttered with build output

Related Question