Disclaimer!
Use this at your own risk. And back up EVERYTHING!
Moreover, consider this as a starting point, since it is far from being perfect.
I have written a small bash script that can do all of your three requisites in a fairly general fashion, however, it is better if you personalize is based on your project structure.
It should be compatible with any standard Unix shells (including OS X). Apart from that, you need latexpand that will take care of inlining your files.
Here I will explain how each part of the script work and how it manages to do what it does. Although I would suggest to use its part separately (instead of running it as a whole). Perhaps, it gives you a better control of what you wanna do.
#!/bin/bash
# Defining some variable for you files:
# 1. The file containing all your macros
# 2. The file that the script uses to contain trimmed (i.e. only used) macros
# 3. Main .tex file
ORIGINAL_MACRO_FILE=macros_original.tex
TRIMMED_MACRO_FILE=macros_original_trimmed.tex
MAIN_FILE=paper.tex
if [[ -f macros_regex.txt || -f macros_regex_trimmed.txt || -f $TRIMMED_MACRO_FILE ]]; then
echo "Some temp files are already here. Please get rid of them first..."
exit 0
fi
# Making a trimmed version of macro files:
echo -ne '\\b(' > macros_regex.txt
echo -ne '\\b(' > macros_regex_trimmed.txt
grep -Po '(?<=\\def\\)(.+?)(?=\{)' $ORIGINAL_MACRO_FILE | tr '\n' '\|' >> macros_regex.txt
sed -i 's/.$/\)\\b/' macros_regex.txt
grep -Porh --exclude=$ORIGINAL_MACRO_FILE $(cat ./macros_regex.txt) *.tex | sort | uniq | tr '\n' '\|' >> macros_regex_trimmed.txt
sed -i 's/.$/\)\\b/' macros_regex_trimmed.txt
grep -P $(cat ./macros_regex_trimmed.txt) $ORIGINAL_MACRO_FILE > $TRIMMED_MACRO_FILE
# Backing up the original macro file:
cp $ORIGINAL_MACRO_FILE $ORIGINAL_MACRO_FILE\_backup
mv $TRIMMED_MACRO_FILE $ORIGINAL_MACRO_FILE
# Inline the file to collect have one big file. This is needed for finding unused macros
perl latexpand --keep-comments $MAIN_FILE > inlined_paper.tex
rm macros_regex.txt macros_regex_trimmed.txt
# Putting back the original macro files:
rm $ORIGINAL_MACRO_FILE
mv $ORIGINAL_MACRO_FILE\_backup $ORIGINAL_MACRO_FILE
# Removing comments
sed -ri '/\%[^\n].+/ d' inlined_paper.tex
# Removing blank lines if you want
sed -ie '/^$/ d' inlined_paper.tex
It needs two files to start:
ORIGINAL_MACRO_FILE=macros_original.tex
MAIN_FILE=paper.tex
The file that contains all your macros and the main tex file. TRIMMED_MACRO_FILE is a temporary file that holds the list of used macros. It then checks if this temp file, as well as two other text files exist and if negative, it continues (note that script will delete these auxiliary files once it is done).
It first solves your second problem! It searches in your macro file using this regular expression (?<=\\def\\)(.+?)(?=\{), and collects the name of all your macros. In my example I assumed macros are in the form of \def\name{..., however, however, if you are using other macro-definition commands here are some regular expressions:
(?<=\\newcommand\{\\)(.+?)(?=\}) for newcommand(?<=\\renewcommand\{\\)(.+?)(?=\}) for renewcommand(?<=\\newcommand\*\{\\)(.+?)(?=\}) for newcommand*
You can use the logical operator or (|) in your regex to have multiple of aforementioned definitions, e.g.
((?<=\\def\\)|(?<=\\renewcommand\{\\))(.+?)(?=\{)
uses both def and renewcommand syntaxes.
It then stores all of macro names in macros_regex.txt in the following form:
\b(amacro|anothermacro|foo|bar|etc)\b
Then using the next line, it checks which of these macros has been used:
grep -Porh --exclude=$ORIGINAL_MACRO_FILE $(cat ./macros_regex.txt) *.tex | sort | uniq | tr '\n' '\|' >> macros_regex_trimmed.txt
This is what happens: grep -Porh means searching in file contents using perl-regex, printing only matching lines, recursive and omit file names. It also excludes your original macro file because obviously it will match with all patterns. Finally we provide the pattern that we created previously by $(cat ./macros_regex.txt) and we search recursively in all tex files.
The results are then sorted and the duplicates are removed ( pipe to sort and uniq respectively). Then again, we create a regex of this output in the form of
\b(anothermacro|foo|etc)\b
But this time it only contains the used macros. Finally, grep this file with original macro files and save the data in TRIMMED_MACRO_FILE. To summarize this, if we have a file in the form of:
% original macros
\def\bfa{{\mbox{\boldmath $a$}}}
\def\bfb{{\mbox{\boldmath $b$}}}
\def\bfc{{\mbox{\boldmath $c$}}}
\def\bfd{{\mbox{\boldmath $d$}}}
\def\bfe{{\mbox{\boldmath $e$}}}
\def\bff{{\mbox{\boldmath $f$}}}
after this stage, we have:
% trimmed macro file
\def\bfb{{\mbox{\boldmath $b$}}}
\def\bfc{{\mbox{\boldmath $c$}}}
\def\bff{{\mbox{\boldmath $f$}}}
that are definition of macros that has been used in your project.
Now, here I explain why I solved your second problem first :D. The idea is that once we have the trimmed version of the file, swap the original and the trimmed macro file and then expand/inline everything. This is done by backing up the original macro file, renaming the trimmed one and finally using latexpand to inline everything.
perl latexpand --keep-comments $MAIN_FILE > inlined_paper.tex
latexpand is a perl program that takes care of \input and \include. As you see I have used the --keep-comments flag to preserve the comments. If you don't do so, it will nicely cleans all the comments, however, this cleaning includes the comments that you have mentioned are needed to be kept. Cleaning the comments is a simple sed one-liner that replaces \%[^\n].+ pattern with blank. That regex means, a percent sign that is not directly followed by a newline but by 1 or more character of any kind. Finally, if you want to remove blank lines, you can use the last command, i.e. sed -ie '/^$/ d' inlined_paper.tex or comment it otherwise.
As you see this is a script that can do the job but it should be customized based on your project structure and commands. Again, I would suggest to use different parts of this code separately instead of running it as whole. For instance, the line that removes the comments is a useful one liner stand-alone.
Finally, I suggest to stick to latexpand as it is a professional tool that is designed for this purpose, instead of this script that I created because my other codes were not compiling (this says a lot!) and I was bored.
P.S. I assumed the reader has a fair familiarity with basic bash commands such as cp, mv and grep. If you find this answer not verbose enough, please leave a comment.
Best Answer
I came up with this little script (ran from the root folder of the project):