[Tex/LaTex] inputenc error with unicode chars and verbatim

fancyvrbinputinput-encodingsunicodeverbatim

I am writing a package where I need to save the contents of several environments in verbatim mode to an auxiliary file. Then the auxiliary file is processed by an external program. In the next compilation of the document, the output of the external program is input in the document.

I am using fancyvrb to output the contents of the environment to the auxiliary file.

I already have something working, except that inputenc gives an error when non ascii unicode characters (i.e. accented letters like á) are used in the environment.

In order to demonstrate the problem, I have written the following simplified package which defines the testenv environment.

\NeedsTeXFormat{LaTeX2e}

\ProvidesPackage{testenv}[2012/03/10 v0.1 JRM test environment]

\RequirePackage{fancyvrb}

% =========================================================
% fancyvrb new commands to append to a file
% =========================================================

\def\VerbatimOutAppend{\FV@Environment{}{VerbatimOutAppend}}

\def\FVB@VerbatimOutAppend#1{%
  \@bsphack
  \begingroup
    \FV@UseKeyValues
    \FV@DefineWhiteSpace
    \def\FV@Space{\space}%
    \FV@DefineTabOut
    \def\FV@ProcessLine{\immediate\write#1}%
    \let\FV@FontScanPrep\relax
    \let\@noligs\relax
    \FV@Scan
}

\def\FVE@VerbatimOutAppend{%
  \endgroup
  \@esphack
}

\DefineVerbatimEnvironment{VerbatimOutAppend}{VerbatimOutAppend}{}

% =========================================================
% my test environment
% =========================================================

\newwrite\testenv@outfile

\newenvironment{testenv}{%
  \immediate\write\testenv@outfile{\noexpand\def\noexpand\testenvcommand\@charlb}%
  \VerbatimEnvironment
  \begin{VerbatimOutAppend}{\testenv@outfile}%
}{%
  \end{VerbatimOutAppend}%
  \immediate\write\testenv@outfile{\@charrb}%
  \ifdefined\testenvcommand\testenvcommand\fi
}

% =========================================================
% final actions
% =========================================================

\AtEndOfPackage{%
  \IfFileExists{\jobname.testenv}{%
    \input{\jobname.testenv}%
  }{%
    \PackageWarning{testenv}{File `\jobname.testenv' not found.}%
  }%
  \immediate\openout\testenv@outfile\jobname.testenv%
}

\AtEndDocument{%
  \closeout\testenv@outfile%
}

\endinput

And I am using the following document to test it:

\documentclass{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{testenv}

\begin{document}

\begin{testenv}
  This is a test with this unicode char: á.
\end{testenv}

\end{document}

When compiled with pdflatex, I am getting the following error message:

! Package inputenc Error: Unicode char \u8:á.\fi not set up for use with LaTeX.


See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              

l.11     \end{testenv}

Does anybody has any clue on how to solve this problem?

Best Answer

You can try changing 

\def\FV@ProcessLine{\immediate\write#1}%

into

\def\FV@ProcessLine{\immediate\unexpandedwrite#1}%

adding the definition

\long\def\unexpandedwrite#1#2{\write#1{\unexpanded{#2}}}

The problem is that characters beyond "7F are made active and they are expanded by \write.

By the way, the line

\immediate\write\testenv@outfile{\noexpand\def\noexpand\testenvcommand\@charlb}%

can be reduced to

\immediate\write\testenv@outfile{\def\string\testenvcommand\@charlb}%

since \def is not expandable.

Related Question