[Tex/LaTex] How to Draw Recurrent Neural Network

tikz-pgf

I googled that tikz is a package for drawing graph like a neural network. I really like to learn how to use it. But I wonder how to draw a recurrent neural network?

rnn

Best Answer

This should help you get started. I created styles for each type of neuron, so you can easily change up the styling and spacing as needed without needing to touch a lot of code. These style definitions are exhaustively commented so you can see what each line does.

\documentclass[border=5pt]{standalone}
\renewcommand{\vec}[1]{\mathbf{#1}} % re-style the vector
\usepackage{tikz}
\usetikzlibrary{
  arrows.meta, % for Straight Barb arrow tip
  fit, % to fit the group box around the central neurons
  positioning, % for relative positioning of the neurons
}

\tikzset{
  neuron/.style={ % style for each neuron
    circle,draw,thick, % drawn as a thick circle
    inner sep=0pt, % no built-in padding between the text and the circle shape
    minimum size=3.5em, % make each neuron the same size regardless of the text inside
    node distance=1ex and 2em, % spacing between neurons (y and x)
  },
  group/.style={ % style for the groups of neurons
    rectangle,draw,thick, % drawn as a thick rectangle
    inner sep=0pt, % no padding between the node contents and the rectangle shape
  },
  io/.style={ % style for the inputs/outputs
    neuron, % inherit the neuron style
    fill=gray!15, % add a fill color
  },
  conn/.style={ % style for the connections
    -{Straight Barb[angle=60:2pt 3]}, % simple barbed arrow tip
    thick, % draw in a thick weight to match other drawing elements
  },
}

\begin{document}
\begin{tikzpicture}
  \node[neuron] (ht1) {$h_t^1$};
  \node[neuron,below=of ht1] (htm) {$h_t^{n-m}$};
  \node[neuron,below=of htm] (htn) {$h_t^n$};
  \node[group,fit={(ht1) (htm) (htn)}] (gr1) {};
  \node[neuron,left=of ht1] (ht-1) {$h_{t-1(2)}^1$};
  \node[io,above=2em of ht1] (yt) {$y_t$};
  \node[io,below=2em of htn] (vt) {$\vec{v}_t$};
  \draw[conn] (ht1) -- (yt);
  \draw[conn] (vt) -- (htn);
  \foreach \destination in {ht1,htm,htn} { % the for loop idea can be expanded to draw the entire diagram quickly
    \draw[conn] (ht-1.east) -- (\destination.west);
  }
\end{tikzpicture}
\end{document}

And here's the result:

enter image description here

OK, that was a lot of stuff! Let's break it down.

Each neuron is drawn as a node with a name in parentheses and the node text in curly braces, for example:

\node[neuron] (ht1)   {$h_t^1$};
%     ^-style  ^-name  ^-node text

Once that first node is placed, all others can be positioned relative to it, with no need for calculating coordinates manually, using the positioning library:

\node[neuron,below=of ht1] (htm) {$h_t^{n-m}$};
%            ^-this node should be placed below the first node (ht1)

The spacing is controlled by the value of the node distance key, which I set in the definition of the neuron style.

The group box around the central neurons is created using the fit library to size a node to fit around a list of existing nodes, like so:

\node[group,fit={(ht1) (htm) (htn)}] (gr1) {};
%     ^-style    ^- list of nodes to fit    ^-empty node text for this one

This node (gr1) has an empty text, because we just want the node to box the list of neurons, we don't need any new text inside it.

Once all the neurons are drawn, we can connect them with \draw statements, which create stroked paths between two coordinates (here, named coordinates) in parentheses. One example:

\draw[conn] (ht1) -- (yt);
%     ^      ^-start  ^-finish
%     `-apply connection style

Drawing all of these individually gets tedious quickly. Fortunately, TikZ offers the concept of for loops, which I used to create the other arrows:

\foreach \destination in {ht1,htm,htn} {
  \draw[conn] (ht-1.east) -- (\destination.west);
}

I added .east and .west as anchors on the nodes we're drawing to and from. This causes the connection to be drawn to that specific point on the node edge. If you leave those off, the arrow will be drawn in a straight line from node center to node center (notice how I left the anchors off in my first example).

The \foreach loop idea can be extended to draw the many neurons in your full diagram efficiently. You can check out TeXample: Neural Network for a nice example of this.

TikZ has excellent documentation, but it might look overwhelming at first glance. If you don't see why I did something, search for the macro name or key name in the manual to see how it works and what it does. You can access the manual on your PC by typing texdoc tikz at a terminal or command prompt. Or, you can access it online at http://texdoc.net/pkg/tikz (warning: it is a large PDF file). Hope this helps you get started! :-)

Related Question