[Tex/LaTex] How to do multiple string replacements

stringstext manipulation

I want to escape URL paths to obtain the percent-escaping that is also applied e.g. by browsers. For this purpose I tried to do the following (out-commented lines are meant to show what I'm aiming here):

\documentclass[a5paper]{article}
\usepackage{xstring}

\begin{document}

\newcommand{\urlescape}[1]{{%
  \def\x{\StrSubstitute{#1}{\%}{\%25}}%
  \def\x{\StrSubstitute{\x}{/}{\%2F}}%
  % \def\x{\StrSubstitute{\x}{\&}{\%26}}%
  % \def\x{\StrSubstitute{\x}{ }{\%20}}%
  % \def\x{\StrSubstitute{\x}{\$}{\%24}}%
  % \def\x{\StrSubstitute{\x}{+}{\%2b}}%
  % \def\x{\StrSubstitute{\x}{,}{\%2c}}%
  % \def\x{\StrSubstitute{\x}{:}{\%3a}}%
  % \def\x{\StrSubstitute{\x}{;}{\%3b}}%
  % \edef\x{\StrSubstitute{\x}{=}{\%3d}}%
  % \def\x{\StrSubstitute{\x}{?}{\%3f}}%
  % \def\x{\StrSubstitute{\x}{@}{\%40}}%
  % \def\x{\StrSubstitute{\x}{"}{\%22}}%
  % \def\x{\StrSubstitute{\x}{<}{\%3c}}%
  % \def\x{\StrSubstitute{\x}{>}{\%3e}}%
  % \def\x{\StrSubstitute{\x}{\#}{\%23}}%
  % \def\x{\StrSubstitute{\x}{\{}{\%7b}}%
  % \def\x{\StrSubstitute{\x}{\}}{\%7d}}%
  % \def\x{\StrSubstitute{\x}{|}{\%7c}}%
  % \def\x{\StrSubstitute{\x}{\^}{\%5e}}%
  % \def\x{\StrSubstitute{\x}{\~}{\%7e}}%
  % \def\x{\StrSubstitute{\x}{[}{\%5b}}%
  % \def\x{\StrSubstitute{\x}{]}{\%5d}}%
  % \def\x{\StrSubstitute{\x}{\`}{\%60}}%
  \x}}

\urlescape{abcd/efgh\%foo\#bar baz}

\end{document}

Error messages

However, that wouldn't work; I get

! Use of \@xs@StrSubstitute@@ doesn't match its definition.
\kernel@ifnextchar ...rved@d =#1\def \reserved@a {
                                                  #2}\def \reserved@b {#3}\f...
l.35 \urlescape{abcd/efgh\%foo\#bar baz}

So I thought well there's that strange \edef thingie from the TeX book so let's give it a try:

\newcommand{\urlescape}[1]{{%
  \def\x{\StrSubstitute{#1}{\%}{\%25}}%
  \edef\x{\StrSubstitute{\x}{/}{\%2F}}%
...

But now I get the following which confuses me:

! Argument of \reserved@a has an extra }.
<inserted text>
                \par
l.35 \urlescape{abcd/efgh\%foo\#bar baz}

Solution and question
I searched and I found several hints where intricate combinations of magic potions à la \noexpand, \unexpanded got thrown into the mix; sadly, I was unable to extract from those solutions an incantation that would work for me (I've probably made it sufficiently clear by now that using TeX is more of an art than a science to me).

Please do not hesitate to point out a package to do just this — URL escaping — but I think that the question of how to perform multiple replacements on strings in LaTeX is of a more general interest and I'm sure I'll need stuff like that quite often.

Update

Explanation: The question has been raised as the above list is so extensive; the simple answer is that I copy-and-pasted these characters from http://perishablepress.com/stop-using-unsafe-characters-in-urls/.

In a more specific use-case, it will be possible and probably desirable to cut down the enumeration, depending on semantics and environment. as for semantics, it will normally make a difference whether you retrieve http://example.com/foo/bar#baz or http://example.com/foo%2Fbar%23baz; whether or not to escape / and/or # in such URLs is, as such, unpredictable, so I started out with a fairly maximal list.

Best Answer

You are defining \x in terms of itself, which is not a good thing to do. You rather want to store the intermediate result in a temporary macro, which xstring allows to do with the trailing optional argument.

Store the argument after doing the first substitution and then act on the stored string. There's also a problem with expansion that has to be taken care of: the best is to suppress xstring expansions and do only one manually:

\documentclass[a5paper]{article}
\usepackage{xstring}

\newcommand{\urlescape}[1]{{%
  \noexpandarg % suppress expansions made by xstring
  \StrSubstitute{#1}{\%}{\%25}[\x]% first step
  \expandafter\StrSubstitute\expandafter{\x}{/}{\%2F}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\&}{\%26}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{ }{\%20}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\$}{\%24}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{+}{\%2b}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{,}{\%2c}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{:}{\%3a}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{;}{\%3b}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{?}{\%3f}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{@}{\%40}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{"}{\%22}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{<}{\%3c}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{>}{\%3e}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\#}{\%23}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\{}{\%7b}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\}}{\%7d}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{|}{\%7c}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\^}{\%5e}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\~}{\%7e}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{[}{\%5b}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{]}{\%5d}[\x]%
  \expandafter\StrSubstitute\expandafter{\x}{\`}{\%60}[\x]%
  \x}}

\begin{document}

\urlescape{abcd/efgh\%foo\#bar baz}

\end{document}

enter image description here

Some syntactic sugar may be better:

\documentclass[a5paper]{article}
\usepackage{xstring}

\newcommand{\urlescapestep}[2]{%
  \expandafter\StrSubstitute\expandafter{\x}{#1}{#2}[\x]%
}
\newcommand{\urlescape}[1]{{%
  \noexpandarg % suppress expansions made by xstring
  \StrSubstitute{#1}{\%}{\%25}[\x]% first step
  \urlescapestep{/}{\%2F}%
  \urlescapestep{\&}{\%26}%
  \urlescapestep{ }{\%20}%
  \urlescapestep{\$}{\%24}%
  \urlescapestep{+}{\%2b}%
  \urlescapestep{,}{\%2c}%
  \urlescapestep{:}{\%3a}%
  \urlescapestep{;}{\%3b}%
  \urlescapestep{?}{\%3f}%
  \urlescapestep{@}{\%40}%
  \urlescapestep{"}{\%22}%
  \urlescapestep{<}{\%3c}%
  \urlescapestep{>}{\%3e}%
  \urlescapestep{\#}{\%23}%
  \urlescapestep{\{}{\%7b}%
  \urlescapestep{\}}{\%7d}%
  \urlescapestep{|}{\%7c}%
  \urlescapestep{\^}{\%5e}%
  \urlescapestep{\~}{\%7e}%
  \urlescapestep{[}{\%5b}%
  \urlescapestep{]}{\%5d}%
  \urlescapestep{\`}{\%60}%
  \x}}

\begin{document}

\urlescape{abcd/efgh\%foo\#bar baz}

\end{document}

Related Solutions

[Tex/LaTex] from string to command

TeX is a macro replacement language. That means that when we do

\def\foo{bar}

TeX simply stores bar as the replacement text for \foo. Thus when we use \foo, TeX replaces it exactly with bar, and carries on working. The content of a macro is not a string: it's a token list: thus we can do

\def\fooa{\foob}
\def\foob{bar}

and see the same outcome (TeX replaces \fooa with \foob,then \foob with bar). LaTeX's \newcommand is a wrapper around \def, so the principal is identical (although \newcommand will complain if \foo already exists, which \def will not).

The \gdef primitive works in the same way as \def except it applies globally (\def is local, obeying TeX's grouping system). Thus

\newcommand{\AuthorNames}[1]{\gdef\@AuthorNames{#1}}

works by defining \@AuthorNames as a macro to be replaced by whatever is given as the argument to \AuthorNames. As mentioned in the comments, this approach means that the user has a simple interface

\AuthorNames{whatever}

rather than needing to know \gdef and why it should be used here.

TeX also has another form of token storage: the token register or 'toks'. There are reasons for picking a toks over a macro for storage in some cases. However, most of the time you can happily use a macro: they are in many ways more convenient.

[Tex/LaTex] General text/string manipulation from within LaTeX

You've stated that you "may want to do things like replace all occurrences of = with >" and also that "[c]oding should ideally by done in the preamble".

I'm going to keep my fingers crossed that you'll reconsider the decision not to use LuaLaTeX. Lua (the programming language) has a very flexible and powerful string library, and LuaTeX offers several ways to assign Lua-coded functions to various "callbacks" -- meeting your requirement that the coding should be all done in the preamble. In the following example, the function eq2gt (which, as its name suggests, replaces all instances of = with >) is assigned to the process_input_buffer callback, which operates at a very early stage of processing, viz., before TeX's "eyes" start their processing. That way, the eq2gt function can act as a pre-processor, modifying parts of the input file "on the fly" before the typesetting job itself commences.

% !TEX TS-program = lualatex
\documentclass{article}

%% Lua-side code
\usepackage{luacode}
\begin{luacode}
function eq2gt ( buff )
   return ( string.gsub ( buff , "=" , ">" ) )
end
\end{luacode}

%% TeX-side code
\AtBeginDocument{\luadirect{luatexbase.add_to_callback(
   "process_input_buffer" , eq2gt , "eq2gt" )}}

\begin{document} 
\[
1+1+1=2    % not correct...
\]

$1-1-1=-2$ % not correct either...
\end{document}