[Tex/LaTex] latexmk specified output directory

bibtexlatexmkmiktex

latexmk version is Latexmk, John Collins, 17 Jan. 2018. Version 4.55, OS is Windows with MiKTeX2.9, console emulator I use cmder, which include perl. The perl version is perl5 (revision 5 version 24 subversion 1)

here is my folder structure:

top_dir
|-main.tex
|-charpter1
  |-chap1.tex
|-charpter2
  |-chap2.tex
|-reference
  |-ref.tex
  |-ref_db.bib

My main.tex looks like:

\documentclass[oneside,a4paper]{book}

\begin{document}

\include{chapter1/chap1}
\include{chapter2/chap2}
\include{reference/ref}

\end{document}

Meanwhile, in ref.tex,:

\clearpage
\bibliographystyle{unsrt}
\bibliography{reference/ref_db}

other files are empty.

I can get every correct by using following command:

latexmk -pdf -bibtex --interaction=nonstopmode -pdflatex="pdflatex %O %S"

However, I want to change all outputs to a output dir, then I change my build command to:

latexmk -pdf -bibtex --interaction=nonstopmode -outdir=build -pdflatex="pdflatex -output-directory=build %O %S"

it do can generate output dir namely build, but it will fail when generate reference, it said:

latexmk -pdf -bibtex -outdir=build --interaction=nonstopmode -pdflatex="pdflatex %O %S"
Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Latexmk: making output directory 'build'
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
Latexmk: I am working around an offset relative to my system time by
   -167 secs for file times in directory 'build/'.
------------
Running 'pdflatex  --interaction=nonstopmode -recorder -output-directory="build"  "main.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.18 (MiKTeX 2.9.6500 64-bit)
entering extended mode
(main.tex
LaTeX2e <2017-04-15>
Babel <3.15> and hyphenation patterns for 75 language(s) loaded.
("C:\Program Files\MiKTeX 2.9\tex\latex\base\book.cls"
Document Class: book 2014/09/29 v1.4h Standard LaTeX document class
("C:\Program Files\MiKTeX 2.9\tex\latex\base\bk10.clo"))
(Y:\Latex\latexmk_test\build\main.aux) (chapter1/chap1.tex) (chapter2/chap2.tex
) (reference/ref.tex
No file main.bbl.
) (Y:\Latex\latexmk_test\build\main.aux
(Y:\Latex\latexmk_test\build\chapter1/chap1.aux)
(Y:\Latex\latexmk_test\build\chapter2/chap2.aux)
(Y:\Latex\latexmk_test\build\reference/ref.aux)) )
No pages of output.
Transcript written on Y:\Latex\latexmk_test\build\main.log.
=== TeX engine is 'pdfTeX'
Latexmk: Non-existent bbl file 'build/main.bbl'
 No file main.bbl.
Latexmk: Log file says no output from latex
Latexmk: For rule 'pdflatex', no output was made
Latexmk: Found bibliography file(s) [reference/ref_db.bib]
Latexmk: applying rule 'bibtex build/main'...
Rule 'bibtex build/main': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'build/main.aux'
   Non-existent destination files:
      'build/main.bbl'
------------
Run number 1 of rule 'bibtex build/main'
------------
For rule 'bibtex build/main', running '&run_bibtex(  )' ...
------------
Running 'bibtex  "main"'
------------
This is BibTeX, Version 0.99d (MiKTeX 2.9.6500 64-bit)
The top-level auxiliary file: main.aux
A level-1 auxiliary file: chapter1/chap1.aux
A level-1 auxiliary file: chapter2/chap2.aux
A level-1 auxiliary file: reference/ref.aux
The style file: unsrt.bst
I couldn't open database file reference/ref_db.bib
---line 3 of file reference/ref.aux
 : \bibdata{reference/ref_db
 :                          }
I'm skipping whatever remains of this command
I found no \citation commands---while reading file main.aux
I found no database files---while reading file main.aux
(There were 3 error messages)
Latexmk: Errors, so I did not complete making targets
Collected error summary (may duplicate other messages):
  bibtex build/main: Bibtex errors: See file 'build/main.blg'
  pdflatex: failed to create output file
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.

Here is the compile pass result without "build" output folder

 latexmk -pdf -bibtex --interaction=nonstopmode -pdflatex="pdflatex %O %S"
Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
Latexmk: I am working around an offset relative to my system time by
   -166 secs for file times in directory ''.
------------
Running 'pdflatex  --interaction=nonstopmode -recorder  "main.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.18 (MiKTeX 2.9.6500 64-bit)
entering extended mode
(main.tex
LaTeX2e <2017-04-15>
Babel <3.15> and hyphenation patterns for 75 language(s) loaded.
("C:\Program Files\MiKTeX 2.9\tex\latex\base\book.cls"
Document Class: book 2014/09/29 v1.4h Standard LaTeX document class
("C:\Program Files\MiKTeX 2.9\tex\latex\base\bk10.clo")) (main.aux)
(chapter1/chap1.tex) (chapter2/chap2.tex) (reference/ref.tex
No file main.bbl.
) (main.aux (chapter1/chap1.aux) (chapter2/chap2.aux) (reference/ref.aux)) )
No pages of output.
Transcript written on main.log.
=== TeX engine is 'pdfTeX'
Latexmk: Non-existent bbl file 'main.bbl'
 No file main.bbl.
Latexmk: Log file says no output from latex
Latexmk: For rule 'pdflatex', no output was made
Latexmk: Found bibliography file(s) [reference/ref_db.bib]
Latexmk: applying rule 'bibtex main'...
Rule 'bibtex main': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'main.aux'
   Non-existent destination files:
      'main.bbl'
------------
Run number 1 of rule 'bibtex main'
------------
For rule 'bibtex main', running '&run_bibtex(  )' ...
------------
Running 'bibtex  "main"'
------------
This is BibTeX, Version 0.99d (MiKTeX 2.9.6500 64-bit)
The top-level auxiliary file: main.aux
A level-1 auxiliary file: chapter1/chap1.aux
A level-1 auxiliary file: chapter2/chap2.aux
A level-1 auxiliary file: reference/ref.aux
The style file: unsrt.bst
I found no \citation commands---while reading file main.aux
Database file #1: reference/ref_db.bib
(There was 1 error message)
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'chapter1/chap1.aux'
      'chapter2/chap2.aux'
      'main.aux'
      'main.bbl'
      'reference/ref.aux'
------------
Run number 2 of rule 'pdflatex'
------------
------------
Running 'pdflatex  --interaction=nonstopmode -recorder  "main.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.18 (MiKTeX 2.9.6500 64-bit)
entering extended mode
(main.tex
LaTeX2e <2017-04-15>
Babel <3.15> and hyphenation patterns for 75 language(s) loaded.
("C:\Program Files\MiKTeX 2.9\tex\latex\base\book.cls"
Document Class: book 2014/09/29 v1.4h Standard LaTeX document class
("C:\Program Files\MiKTeX 2.9\tex\latex\base\bk10.clo")) (main.aux
(chapter1/chap1.aux) (chapter2/chap2.aux) (reference/ref.aux))
(chapter1/chap1.tex) (chapter2/chap2.tex) (reference/ref.tex (main.bbl

LaTeX Warning: Empty `thebibliography' environment on input line 3.

)) [1{C:/Users/haochen/AppData/Local/MiKTeX/2.9/pdftex/config/pdftex.map}]
(main.aux (chapter1/chap1.aux) (chapter2/chap2.aux) (reference/ref.aux)) )<C:/P
rogram Files/MiKTeX 2.9/fonts/type1/public/amsfonts/cm/cmbx12.pfb><C:/Program F
iles/MiKTeX 2.9/fonts/type1/public/amsfonts/cm/cmr10.pfb>
Output written on main.pdf (1 page, 17996 bytes).
Transcript written on main.log.
=== TeX engine is 'pdfTeX'
Latexmk: Found input bbl file 'main.bbl'
Latexmk: Log file says output to 'main.pdf'
Latexmk: Found bibliography file(s) [reference/ref_db.bib]
Latexmk: applying rule 'bibtex main'...
Rule 'bibtex main': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'reference/ref.aux'
------------
Run number 2 of rule 'bibtex main'
------------
For rule 'bibtex main', running '&run_bibtex(  )' ...
------------
Running 'bibtex  "main"'
------------
This is BibTeX, Version 0.99d (MiKTeX 2.9.6500 64-bit)
The top-level auxiliary file: main.aux
A level-1 auxiliary file: chapter1/chap1.aux
A level-1 auxiliary file: chapter2/chap2.aux
A level-1 auxiliary file: reference/ref.aux
The style file: unsrt.bst
I found no \citation commands---while reading file main.aux
Database file #1: reference/ref_db.bib
(There was 1 error message)
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'reference/ref.aux'
------------
Run number 3 of rule 'pdflatex'
------------
------------
Running 'pdflatex  --interaction=nonstopmode -recorder  "main.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.18 (MiKTeX 2.9.6500 64-bit)
entering extended mode
(main.tex
LaTeX2e <2017-04-15>
Babel <3.15> and hyphenation patterns for 75 language(s) loaded.
("C:\Program Files\MiKTeX 2.9\tex\latex\base\book.cls"
Document Class: book 2014/09/29 v1.4h Standard LaTeX document class
("C:\Program Files\MiKTeX 2.9\tex\latex\base\bk10.clo")) (main.aux
(chapter1/chap1.aux) (chapter2/chap2.aux) (reference/ref.aux))
(chapter1/chap1.tex) (chapter2/chap2.tex) (reference/ref.tex (main.bbl

LaTeX Warning: Empty `thebibliography' environment on input line 3.

)) [1{C:/Users/haochen/AppData/Local/MiKTeX/2.9/pdftex/config/pdftex.map}]
(main.aux (chapter1/chap1.aux) (chapter2/chap2.aux) (reference/ref.aux)) )<C:/P
rogram Files/MiKTeX 2.9/fonts/type1/public/amsfonts/cm/cmbx12.pfb><C:/Program F
iles/MiKTeX 2.9/fonts/type1/public/amsfonts/cm/cmr10.pfb>
Output written on main.pdf (1 page, 17996 bytes).
Transcript written on main.log.
=== TeX engine is 'pdfTeX'
Latexmk: Found input bbl file 'main.bbl'
Latexmk: Log file says output to 'main.pdf'
Latexmk: Found bibliography file(s) [reference/ref_db.bib]
Latexmk: All targets (main.pdf) are up-to-date

Best Answer

The problem arose because the OP was using msys and its Perl together with a native MS-Windows implementation of bibtex (in MiKTeX).

There are two solutions

Update to latexmk 4.56 (which will be on CTAN as soon as the files propagate to the mirror sites, and will then appear in the standard distributions, TeXLive and MiKTeX). It's also available at http://personal.psu.edu/jcc8/latexmk/
Install a native MS-Windows Perl, and ensure that gets used to run latexmk.

Under the hood details (which may be of use to users with related problems): Msys and MinGW give a Unix emulation layer on top of MS-Windows; from latexmk's point-of-view there's effectively a mixture of operating systems, MS-Windows and Unix, particularly as regards names of files. There's some trickery that latexmk goes through to persuade bibtex to work when there is (a) an output directory, (b) included files in the source document, and (c) a .bib file is in the document directory or a subdirectory. The trickery involves a temporary change of directory with a setting of BIBINPUTS and BSTINPUTS. Msys's Perl provides the name of the current directory in Unix form, but it needs to be translated (quite trivially) to MS-Windows form to be useful for bibtex. It turns out latexmk already had a solution to the same problem for cygwin, so once the diagnosis had been made (off-line), the fix was easy.

Related Solutions

[Tex/LaTex] TexStudio + Bibtex + build/auxiliary folder

Assuming the folder structure in the question, you should have the following configs.

Under Options > Configure TexStudio ...> Commands

PdfLatex: pdflatex.exe -synctex=1 -interaction=nonstopmode %.tex -aux-directory=./auxiliary/
BibTex: bibtex.exe ./auxiliary/%

Under Options > Configure TexStudio ...> Build (after you checked Show Advanced Options on the bottom-left corner):

Add ./auxiliary/ into the Log File text-box in the lower pane with the Build Options.

Note: this does not work if you use external applications such as gnuplot. The external application won't be able to find the required build files now present outside of the root directory.

[Tex/LaTex] Bibtex finished with 3 error messages

You asked,

>This is the route to follow, isn't it?
>    compile main.tex with pdflatex       [this generates main.aux]
>    launch BibTex on main                [this generates main.bbl]
>    compile main.tex with pdflatex twice [this incorporates main.bbl into the pdf file]

This would indeed be the route to follow if your document were using BibTeX. However, since your document uses the biblatex package with the option backend=biber, you should be running biber, not BibTeX. For your document, then, the correct sequence is:

> run pdfLaTeX on main[.tex]
> run biber on main
> run pdfLaTeX on main

Incidentally, you should replace the deprecated instruction \bibliography{articles} with

\addbibresource{articles.bib}

Two comments about the linespread choice: (a) Don't manipulate the low-level TeX parameter \linespread directly. Instead, load the setspace package and issue the directive \setstretch{1.5}. (b) One indeed does encounter the recommendation that leading should be slightly looser with a Palatino font. However, that recommendation is generally for a factor around 1.04 or 1.05, never 1.5! If you absolute must increase the leading, by all means to do. Just don't claim you're doing so because Palatino somehow requires it...

\RequirePackage{filecontents}
\begin{filecontents}{articles.bib}
@misc{a:3001,
  author = "Anne Author",
  title  = "Initial Thoughts",
  year   = 3001,
}
@misc{z:3002,
  author = "Zo{\"e} Zwicky",
  title  = "Final Thoughts",
  year   = 3002,
}
\end{filecontents}

\documentclass[11pt, a4paper, twoside]{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}


\usepackage[margin=2.5cm]{geometry}
\usepackage[english]{babel}
\usepackage[babel]{csquotes}
\usepackage[useprefix, style=numeric-comp, sorting=nyt, 
            backend=biber, hyperref=true]{biblatex}
\addbibresource{articles.bib}

\usepackage{fancyhdr} %with "twoside" \documentclass option!!
\pagestyle{fancy}
\fancyhf{}
\fancyhead[LE,RO]{\thepage} % Left if Even, Right if Odd pg
\fancyhead[RE,LO]{Urban Gardens}
\fancyfoot[]{} %
\fancyfoot[]{}


% FONT CHOICE
% Palatino for serif & math, Helvetica for ss, Courier for tt
\usepackage{mathpazo} % math & rm

%%%\linespread{1.50}     % Palatino needs more leading (space between lines)
% Don't set the low-level parameter "\linespread" directly!
% Instead, load the "setspace" package and state "\setstretch{1.5}"
\usepackage{setspace}
\setstretch{1.5}

\usepackage[scaled]{helvet} % ss
\usepackage{courier} % tt
\normalfont

\usepackage{graphicx}
\usepackage{booktabs}
% \usepackage{subcaption} % \begin{subfigure} within 'figure' environment
\usepackage{lipsum}
\usepackage{keyval}
\usepackage{subfig}
\usepackage{caption}
\usepackage[colorlinks=false, pdfborder={0 0 0}]{hyperref}
\usepackage[capitalize, italian]{cleveref}

\frenchspacing

% \graphicspath{{./img/}}

\title{My Essay}
\author{WW}
\date{}

\begin{document}

\maketitle{}

\tableofcontents

\bigskip
\hrule
\bigskip

\lipsum[2]

%\clearpage
\phantomsection
\addcontentsline{toc}{section}{\refname}

\nocite{*}
\printbibliography

\end{document}
%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End: