[Tex/LaTex] Syncing Python code with LaTeX document using Listings package

Question

I have a Tex document that uses the listings package to print a Python source code. I also want to separately print certain Python function signatures in my document. I do not want to use the line number functionality from the listings package since the Python code might be edited in the future.

Is there any way to automatically sync certain parts of the Python code with the Tex document (without using line numbers)?

Thank you very much!

Answer 1

The main problem is that you need to analyze the Python source code, which TeX can't (easily) do. Luckily Python is capable of analyzing itself, using abstract syntax trees (see also some more thorough documentation). The idea is that we have the Python ast analyze the code and write what it finds to a TeX file.

#!/usr/local/bin/python3

import ast

""" The python file we want to analyze.  Happens to be itself """
pythonfilename = 'pythonlinenumbers.py'

newcommands = []

def makenewcommand(command,
                   output):
    """ Turns the command and line number into the appropriate command.
    The signature is split onto two lines to make it more complicated.
    We have to play tricks with the trailing \, because we can't end a string
    with a single backslash. """
    return r'\newcommand{''\\'+command+'}{'+str(output)+'}\n'

class FuncLister(ast.NodeVisitor):
    def visit_FunctionDef(self, node):
        """ Recursively visit all functions, determining where each function
        starts, where its signature ends, and where the function ends.  Store
        these in the TeX variables \firstline@funcname, \sigend@funcname,
        \docend@funcname, and \lastline@funcname. """
        newname=node.name.replace('_','@') # _ isn't allowed in a TeX command
        newcommands.append(makenewcommand('firstline@'+newname,node.lineno))
        sigend = max(node.lineno,lastline(node.args))
        newcommands.append(makenewcommand('sigend@'+newname,sigend))
        docstring = ast.get_docstring(node)
        docstringlength = len(docstring.split('\n')) if docstring else -1
        newcommands.append(makenewcommand('docend@'+newname,sigend+docstringlength))
        newcommands.append(makenewcommand('lastline@'+newname,lastline(node)))
        self.generic_visit(node)

def lastline(node):
    """ Recursively find the last line of a node """
    return max( [ node.lineno if hasattr(node,'lineno') else -1 , ]
                +[lastline(child) for child in ast.iter_child_nodes(node)] )

with open(pythonfilename) as f:
    code = f.read()
FuncLister().visit(ast.parse(code))
with open('linenumbers.tex','w') as f:
    for newcommand in newcommands:
        f.write(newcommand)

Creates the file linenumbers.tex:

\newcommand{\firstline@makenewcommand}{10}
\newcommand{\sigend@makenewcommand}{11}
\newcommand{\docend@makenewcommand}{15}
\newcommand{\lastline@makenewcommand}{16}
\newcommand{\firstline@visit@FunctionDef}{19}
\newcommand{\sigend@visit@FunctionDef}{19}
\newcommand{\docend@visit@FunctionDef}{23}
\newcommand{\lastline@visit@FunctionDef}{32}
\newcommand{\firstline@lastline}{34}
\newcommand{\sigend@lastline}{34}
\newcommand{\docend@lastline}{35}
\newcommand{\lastline@lastline}{37}

Then, you can have your main TeX file input the derived TeX file, and use what it found:

\documentclass{article}

\usepackage{listings}

\immediate\write18{python3 pythonlinenumbers.py}
\makeatletter
\input{linenumbers}
\newcommand{\showfunc}[1]{%
 #1 signature:
 \lstinputlisting[
    firstline=\csname firstline@#1\endcsname,
    lastline=\csname sigend@#1\endcsname,
    language=Python]
 {pythonlinenumbers.py}

 #1 in its entirety:
 \lstinputlisting[
    firstline=\csname firstline@#1\endcsname,
    lastline=\csname lastline@#1\endcsname,
    language=Python]
 {pythonlinenumbers.py}}
\makeatother

\begin{document}

Here's the lastline function:

\showfunc{lastline}

\bigskip

Here's the makenewcommand function (also has a multi-line signature):

\showfunc{makenewcommand}

\end{document}

This results in