[Tex/LaTex] A question on tikz/pgf externalization, standalone, and includeonly for compiling large document

includestandalonetikz-externaltikz-pgf

I have a goal to build an efficient compiling environment to build large documents. The requirements are these:

Use tikz/pgf to draw many figures;
Use \includeonly to build only a portion of the large document at a time.
Make the compilation to only rebuild the modified tikz/pgf figures.

I know how to make tikz/pgf figures to be built using the externalized mode. However, the externalized mode does not work with \includeonly.

Moreover, I have seen some posts on rebuilding the modified tikz/pgf figures using the standalone package. Some posts even said that it is better not to use tikz/pgf externalized mode and the standalone package together. I am quite confused on this claim. Can anyone do a clarification on the joint use of tikz/pgf externalized mode and the standalone package?

I would like to ask the experts for existing tutorials or examples on making tikz/pgf externalized mode to work with \includeonly.

Below is a minimal working example.

main.tex

\documentclass[\myopts]{article}

\usepackage{tikz}
\usepackage{pgfplots}
\usepackage{pgfplotstable}
\usepackage{tikzscale}

\usetikzlibrary{external}
\tikzexternalize
%\tikzset{external/mode=list and make}
%\tikzset{external/check=diff}
\tikzset{external/force remake}
%\tikzsetexternalprefix{figure-build/, up to date check=md5, force remake}

\pgfplotsset{compat=1.8}
%\pgfplotsset{compat=newest} % necessary for new features

\usepackage{ifpdf}

\ifpdf
  \tikzset{external/system call={%
    pdflatex \tikzexternalcheckshellescape -halt-on-error -interaction=batchmode
      -jobname "\image" "\string\def\string\myopts{\myopts}\texsource"}}
\else
    \tikzset{external/system call={latex \tikzexternalcheckshellescape -halt-on-error -interaction=batchmode -jobname "\image" "\string\def\string\myopts{\myopts}\texsource"; dvips -E -o "\image".eps "\image".dvi; ps2pdf "\image".eps "\image".pdf;}}
\fi

\usepackage{import}
\usepackage{filecontents}

\begin{filecontents}{datafile}
-0.000632
0.004692
0.001407
0.005768
0.003117
0.001605
0.001742
-0.000874
-0.000973
-0.000436
-0.001578
-0.001953
-0.004275
-0.001585
-0.000957
-0.000819
-0.000924
-0.003559
0.001949
-0.000300
-0.003085
-0.002050
-0.000493
-0.001378
-0.000970
0.000912
0.001667
-0.000974
-0.000592
-0.000865
-0.000933
0.000435
-0.001278
0.001508
-0.000566
-0.003608
0.002768
0.005676
0.000331
-0.000527
-0.000589
-0.000751
-0.000503
-0.000718
-0.000950
-0.005616
-0.004347
-0.001818
-0.000503
0.001063
0.002109
-0.004156
-0.002612
-0.000773
-0.000635
-0.000218
-0.001041
-0.001247
-0.000283
-0.004434
-0.000370
0.002669
0.001302
0.001351
0.001618
0.002028
0.001291
0.004341
0.001141
0.001058
-0.004537
-0.000123
-0.000052
0.000010
-0.000643
0.001403
-0.000751
-0.000201
-0.001639
-0.000316
-0.000757
-0.000872
0.000530
-0.000969
-0.000826
-0.001076
-0.000808
0.001687
0.001356
0.001870
0.001016
0.002165
0.005372
0.001128
0.002269
0.001878
0.000839
-0.000798
-0.000846
0.000146
\end{filecontents}

\begin{filecontents}{histogram.tikz}
\tikzsetnextfilename{Fig-Histogram}
\begin{tikzpicture}
    \begin{axis}[
        xmode=linear,
        ymode=linear,
        axis x line*=bottom,
        axis y line*=left,
        tick label style={font=\small},
        grid=both,
        tick align=outside, 
        tickpos=left,
        xlabel=Bins,
        ylabel=Count,
        width=0.45\textwidth,
        height=0.4\textwidth,
    ]
    \addplot+[raw gnuplot, ybar, bar width=0.002, color=red] gnuplot {
        binwidth=0.005;
        bin(x,bw)=bw*floor(x/bw);
        plot "datafile.tex" using (bin($1,binwidth)):(1.0) smooth freq with boxes;
    };
    \end{axis}
\end{tikzpicture}
\end{filecontents}

\begin{filecontents}{mainbody.tex}
\begin{figure}[bth]
    \begin{tabular}{p{3in}}
        \includegraphics[]{histogram}
    \end{tabular}
    \caption{\small{Histogram.}}
\end{figure}
\end{filecontents}

\begin{filecontents}{doc1.tex}
\begin{figure}[bth]
    \begin{tabular}{p{3in}}
        \includegraphics[]{histogram}
    \end{tabular}
    \caption{\small{Histogram.}}
\end{figure}
\end{filecontents}

\begin{filecontents}{doc2.tex}
\begin{figure}[bth]
    \begin{tabular}{p{3in}}
        \includegraphics[]{histogram}
    \end{tabular}
    \caption{\small{Histogram.}}
\end{figure}
\end{filecontents}

\includeonly{\myfilename}

\begin{document}
\setcounter{page}{1}
\subincludefrom{./}{mainbody}
\setcounter{page}{1}
\subincludefrom{./}{doc1}
\setcounter{page}{1}
\subincludefrom{./}{doc2}
\end{document}

I created a Makefile as below.

allinone:
    pdflatex -shell-escape "\def\myopts{11pt}\def\myfilename{mainbody,doc1,doc2}\input{main}"

doc1:
    pdflatex -shell-escape "\def\myopts{11pt}\def\myfilename{doc1}\input{main}"

doc2:
    pdflatex -shell-escape "\def\myopts{11pt}\def\myfilename{doc2}\input{main}"

mainbody:
    pdflatex -shell-escape "\def\myopts{11pt}\def\myfilename{mainbody}\input{main}"

If all components are compiled, then run

make allinone

If only a single component is compiled, the run

make doc1
make doc2
make maninbody

I can not make a correct compile. Please help.

Best Answer

I know I'm a little bit late, but I have encountered the same problem.

The easiest solution is to set a different filename prefix for each part you want to standalone-compile. Hence, the counter used for numbering the externalized figures will be reset at each new part.

Here's an example:

\includeonly{Chapter1,Chapter2} % or whatever

{\tikzsetfigurename{External-TikZ/Chap1_}
    \include{Chapter1}
}
{\tikzsetfigurename{External-TikZ/Chap2_}
    \include{Chapter2}
}
{\tikzsetfigurename{External-TikZ/Chap3_}
    \include{Chapter3}
}

All options for the externalization procedure are detailed in the PGF manual (pp. 615-625) Regards.

Related Solutions

[Tex/LaTex] Correct way to use include and includeonly when writing a large document like a thesis

Cross referencing and citation should work fine with \input. input is more or less invisible to the TeX processing, it just allows you to split up the file into smaller units for ease of editing.

If you do not use \includeonly then \include{file} is more or less the same as \clearpage\input{file}\clearpage so like \input but with forced page breaks. Since your chapter head probably forces a page break anyway this means that switching between \input and \include should not affect output.

If you use \include then you can use \includeonly to speed up processing by just typesetting specified chapters. Note however that it will use old aux files for the other chapters to get the numbering more or less correct. Thus you should occasionally (and initially) process the file without \includeonly so that all chapters are processed and usable aux files are generated for each chapter.

As noted by @Corentin in the comment, in the case of \include it is important that you use the file name without .tex at the end: \include{file1} not \include{file1.tex}.

[Tex/LaTex] Tikz externalization and directory questions

I hope I fully understood what you requested. Here goes.

\tikzexternalize accepts a prefix parameter which tells pdflatex where to store the externalised graphics. So even if you use \input in main.tex to include exercises with tikzpictures, you can set prefix=<something> in main.tex to let pdflatex know that externalised graphics should be stored in that directory – specifically, you want them in the directory of the exercise. You can also separate the code from the processing by saving the externalised graphics in a sub-directory where all exercises of a chapter are stored. Here I use external as the directory.

% Macro holding the exercises directory name (in main.tex).
\newcommand{\exercisesdirectory}{aclass/achapter/exercises/}
% Macro holding the externalized sub-directory (in main.tex).
\newcommand{\externaldirectory}{aclass/achapter/exercises/external/}
\usepackage{tikz}
\usetikzlibrary{external}
% All externalized graphics go go the \externaldirectory
\tikzexternalize[prefix=\externaldirectory]
% Externalise only on-demand.
\tikzexternaldisable

There is a caveat, however: if your main file is not always named main.tex, then the external library will re-externalise graphics with a new filename (I believe the format is \jobname-figureX where X is a running counter starting from 0. This is not what you want; you want to avoid re-externalisation. Thankfully, this issue can be easily circumvented by letting pdflatex know what the externalised graphic should be named using \tikzsetnextfilename:

\tikzexternalenable
\tikzsetnextfilename{ex1fig1} % This graphic will always be named ex1fig1.
\begin{tikzpicture}
  % Move along. Nothing to see here.
\end{tikzpicture}%
\tikzexternaldisable

This means that if this exercise is \input in a file main.tex and a file file.tex compiled with pdflatex -shell-escape, the external library will only create the figure if ex1fig1.pdf does not exist (or if something has changed in the tikzpicture environment etc.)

With all that in mind, your main .tex file would look like this:

`main.tex`

\documentclass{article}

% The path to the exercises.
\newcommand{\exercisesdirectory}{aclass/achapter/exercises/}
% The path where externalised graphics will be stored.
\newcommand{\externaldirectory}{aclass/achapter/exercises/external/}
% Will be used to store the path to the exercise to be processed.
\newcommand{\pathtoexercise}{}

\usepackage{tikz,pgfplots}
\usetikzlibrary{external}
% All auxiliary files and externalised graphics go to \externaldirectory.
\tikzexternalize[prefix=\externaldirectory]
% Only externalise on-demand.
\tikzexternaldisable

% In case you want to include externalised graphics directly.
\graphicspath{ {\externaldirectory} }

\begin{document}
% This will combine the \exercisesdirectory and "firstexercise.tex" strings,
% and store them in \pathtoexercise.
\expandafter\def\expandafter\pathtoexercise\expandafter{\exercisesdirectory firstexercise.tex}
% Then we simply input \pathtoexercise.
\input{\pathtoexercise}
\end{document}

Where firstexercise.tex is stored in aclass/achapter/exercises/.

`firstexercise.tex`:

This is some text from the first exercise. Below are some cool graphics.

\tikzexternalenable
\tikzsetnextfilename{ex1fig1}
\begin{tikzpicture}
  \begin{axis}[%
    scale only axis,
    width=5cm,
    height=5cm,
  ]

    \addplot[red,samples=10] {rnd};

  \end{axis}
\end{tikzpicture}%
\tikzexternaldisable

If you want to include only externalised graphics instead of the whole firstexercise.tex, all you need to do is to set the graphicspath in main.tex to the \externaldirectory, like so: \graphicspath{ {\externaldirectory} }. Thereafter, you simply use \includegraphics with the figure filename, like so:

% \includegraphics will automatically search in the graphicspath directory.
\begin{figure}
\centering
\includegraphics{ex1fig1.pdf} % This is actually in aclass/achapter/exercises/external/ex1fig1.pdf
\caption{Exercise $1$, Figure $1$}
\label{fig:ex1fig1}
\end{figure}

I hope this makes at least some sense and that it is in fact what you requested.