[Tex/LaTex] How to add bias and weight to neural network diagram

diagramsdrawtikz-pgf

I have the following code that I came out with help of this community, but it seems to be missing some elements that I still want to add, like the bias as a node and the weight as something written or a node, I don't know the best way to show it.

\documentclass{standalone}
\usepackage{tikz}
\begin{document}
\pagestyle{empty}
\def\layersep{3cm}
\def\nodeinlayersep{1.5cm}
\begin{tikzpicture}[
   shorten >=1pt,->,
   draw=black!50,
    node distance=\layersep,
    every pin edge/.style={<-,shorten <=1pt},
    neuron/.style={circle,fill=black!25,minimum size=17pt,inner sep=0pt},
    input neuron/.style={neuron, fill=green!50},
    output neuron/.style={neuron, fill=red!50},
    hidden neuron/.style={neuron, fill=blue!50},
    annot/.style={text width=4em, text centered}
]
    % Draw the input layer nodes
    \foreach \name / \y in {1,...,3}
    % This is the same as writing \foreach \name / \y in {1/1,2/2,3/3,4/4}
    \node[input neuron, pin=left:Input \#\y] (I-\name) at (0,-\y-2.5) {};  %%% <-- MODIFIED
    % set number of hidden layers
    \newcommand\Nhidden{2}

    % Draw the hidden layer nodes
    \foreach \N in {1,...,\Nhidden} {
       \foreach \y in {1,...,5} { %%% MODIFIED (1,...,12 -> 1,...,5, and the next five lines)
         \ifnum \y=4
           \node at (\N*\layersep,-\y*\nodeinlayersep) {$\vdots$};
         \else
           \node[hidden neuron] (H\N-\y) at (\N*\layersep,-\y*\nodeinlayersep ) {$\frac{1}{1+e^{-x}}$};
         \fi
       }
    \node[annot,above of=H\N-1, node distance=1cm] (hl\N) {Hidden layer \N};
    }
    % Draw the output layer node
    \node[output neuron,pin={[pin edge={->}]right:Output}, right of=H\Nhidden-3] (O) {}; %%% <-- MODIFIED (from H\Nhidden-6 to H\Nhidden-3) 
    % Connect every node in the input layer with every node in the
    % hidden layer.
    \foreach \source in {1,...,3}
        \foreach \dest in {1,...,3,5} %%% <-- MODIFIED (1,...,12 -> 1...,3,5)
            \path (I-\source) edge (H1-\dest);
    % connect all hidden stuff
    \foreach [remember=\N as \lastN (initially 1)] \N in {2,...,\Nhidden}
       \foreach \source in {1,...,3,5} %%% <-- MODIFIED (1,...,12 -> 1...,3,5)
           \foreach \dest in {1,...,3,5} %%% <-- MODIFIED (1,...,12 -> 1...,3,5)
               \path (H\lastN-\source) edge (H\N-\dest);
    % Connect every node in the hidden layer with the output layer
    \foreach \source in {1,...,3,5} %%% <-- MODIFIED (1,...,12 -> 1...,3,5)
        \path (H\Nhidden-\source) edge (O);

    % Annotate the layers
    \node[annot,left of=hl1] {Input layer};
    \node[annot,right of=hl\Nhidden] {Output layer};
\end{tikzpicture}
% End of code
\end{document}

UPDATE: The result is something like these two questions already answered, which I couldn't do the same.

1) Neural Network illustration

2) drawing a neural network with bias with TikZ

UPDATE 2: I made an example of what I am trying too achieve:

Best Answer

sidenote:

Please add always a minimal working example MWE.
It would be good if you add an image of the result so that people can see directly, what you are expected to achieve. Images of other questions or answers are not helpful. A link is enough.
Please keep in mind to clarify your specific problem or add additional details to highlight exactly what you need. See the How to Ask page for help clarifying questions.

Update answer refer to your comments + improved question:

MWE (update version):

\documentclass{standalone}
\usepackage{tikz}
\begin{document}
\pagestyle{empty}
\def\layersep{3cm}
\def\nodeinlayersep{1.5cm}
\begin{tikzpicture}[
   shorten >=1pt,->,
   draw=black!50,
    node distance=\layersep,
    every pin edge/.style={<-,shorten <=1pt},
    neuron/.style={circle,fill=black!25,minimum size=17pt,inner sep=0pt},
    input neuron/.style={neuron, fill=green!50,},
    output neuron/.style={neuron, fill=red!50},
    hidden neuron/.style={neuron, fill=blue!50},
    annot/.style={text width=4em, text centered},
    bias/.style={neuron, fill=yellow!50,minimum size=4em},%<-- added %%%
]
    % Draw the input layer nodes
    \foreach \name / \y in {1,...,3}
    \node[input neuron, pin=left:Input \#\y] (I-\name) at (0,-\y-2.5) {};  
    % set number of hidden layers
    \newcommand\Nhidden{2}

    % Draw the hidden layer nodes
    \foreach \N in {0,...,\Nhidden} {
       \foreach \y in {0,...,5} { % <-- added 0 instead of 1 %%%%%
     \ifnum \y=4
     \ifnum \N>0 %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
       \node at (\N*\layersep,-\y*\nodeinlayersep) {$\vdots$};
       \else\fi %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     \else
         \ifnum \y=0 %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
         \ifnum \N<3 %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
           \node[bias] (H\N-\y) at (\N*\layersep,-\y*\nodeinlayersep ) {Bias}; %<-- added
           \else\fi %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
         \else %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
            \ifnum \N>0 %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%
           \node[hidden neuron] (H\N-\y) at (\N*\layersep,-\y*\nodeinlayersep ) {$\frac{1}{1+e^{-x}}$}; %<-- added %%%%%%%%%%%
               \else\fi %<-- added %%%%%%%%%%%%
         \fi %<-- added %%%%%%%
         \fi
       }
       \ifnum \N>0 %<-- added %%%%%%
    \node[annot,above of=H\N-1, node distance=1cm,yshift=2cm] (hl\N) {Hidden layer \N}; % <- added yshift=2cm %%%%%%%%%%%%
    \else\fi %<-- added %%%%%
    }
    % Draw the output layer node
    \node[output neuron,pin={[pin edge={->}]right:Output}, right of=H\Nhidden-3] (O) {}; 
    % Connect every node in the input layer with every node in the
    % hidden layer.
    \foreach \source in {1,...,3}
        \foreach \dest in {1,...,3,5} {
          % \path[yellow] (H-0) edge (H1-\dest);
          \path[dashed,orange] (H0-0) edge (H1-\dest); %<-- added %%%%%
            \path[yellow] (I-\source) edge (H1-\dest);};

    % connect all hidden stuff
    \foreach [remember=\N as \lastN (initially 1)] \N in {2,...,\Nhidden}
       \foreach \source in {0,...,3,5} 
           \foreach \dest in {1,...,3,5}{

               \ifnum \source=0 %<-- added %%%%%%%%%%%%%%%%%%%%%%%
           \path[dashed,red](H\lastN-\source) edge (H\N-\dest);%<-- added 
              \else %<-- added %%%
              \path[blue] (H\lastN-\source) edge (H\N-\dest);%<-- added 
              \fi %<-- added %%%
              }; %<-- added %%%%

    % Connect every node in the hidden layer with the output layer
    \foreach \source in {1,...,3,5}
    \path[green] (H\Nhidden-\source) edge (O);
    \path[dashed,red] (H2-0) edge (O); %<-- added %%%%
    % Annotate the layers
    \node[annot,left of=hl1] {Input layer};
    \node[annot,right of=hl\Nhidden] {Output layer};  
\end{tikzpicture}
% End of code
\end{document}

Just a short starting point!

Just to help you adapting the code. A starting point could be the following. I hope it will help you to see the real problem in understanding your code. I added a new row (called \y=0) and added in column \N=2 ( is Hidden layer 2) a new node (\node[bias]). In general the option minimum size=<..> could be helpful for all node styles, see bias/.style={.... The headers are shifted with yshift=2cm. Dashed arrow from bias to output: \path[dashed] (H2-0) edge (O);. All changes are marked with %<-- added %%%%%%.

MWE (starting point):

\documentclass{standalone}
\usepackage{tikz}
\begin{document}
\pagestyle{empty}
\def\layersep{3cm}
\def\nodeinlayersep{1.5cm}
\begin{tikzpicture}[
   shorten >=1pt,->,
   draw=black!50,
    node distance=\layersep,
    every pin edge/.style={<-,shorten <=1pt},
    neuron/.style={circle,fill=black!25,minimum size=17pt,inner sep=0pt},
    input neuron/.style={neuron, fill=green!50,},
    output neuron/.style={neuron, fill=red!50},
    hidden neuron/.style={neuron, fill=blue!50},
    annot/.style={text width=4em, text centered},
    bias/.style={neuron, fill=yellow!50,minimum size=4em},%<-- added %%%
]
    % Draw the input layer nodes
    \foreach \name / \y in {1,...,3}
    \node[input neuron, pin=left:Input \#\y] (I-\name) at (0,-\y-2.5) {};  
    % set number of hidden layers
    \newcommand\Nhidden{2}

    % Draw the hidden layer nodes
    \foreach \N in {1,...,\Nhidden} {
       \foreach \y in {0,...,5} { % <-- added 0 instead of 1 %%%%%%%%%%%%%%%
         \ifnum \y=4
           \node at (\N*\layersep,-\y*\nodeinlayersep) {$\vdots$};
         \else
         \ifnum \y=0 %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
         \ifnum \N=2 %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
           \node[bias] (H\N-\y) at (\N*\layersep,-\y*\nodeinlayersep ) {Bias}; %<-- added
           \else\fi %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
         \else %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
           \node[hidden neuron] (H\N-\y) at (\N*\layersep,-\y*\nodeinlayersep ) {$\frac{1}{1+e^{-x}}$}; %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
         \fi %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
         \fi
       }
    \node[annot,above of=H\N-1, node distance=1cm,yshift=2cm] (hl\N) {Hidden layer \N}; % <- added yshift=2cm %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    }
    % Draw the output layer node
    \node[output neuron,pin={[pin edge={->}]right:Output}, right of=H\Nhidden-3] (O) {}; 
    % Connect every node in the input layer with every node in the
    % hidden layer.
    \foreach \source in {1,...,3}
        \foreach \dest in {1,...,3,5} 
            \path (I-\source) edge (H1-\dest);
    % connect all hidden stuff
    \foreach [remember=\N as \lastN (initially 1)] \N in {2,...,\Nhidden}
       \foreach \source in {1,...,3,5} 
           \foreach \dest in {1,...,3,5}
               \path (H\lastN-\source) edge (H\N-\dest);
    % Connect every node in the hidden layer with the output layer
    \foreach \source in {1,...,3,5}
        \path (H\Nhidden-\source) edge (O);

    % Annotate the layers
    \node[annot,left of=hl1] {Input layer};
    \node[annot,right of=hl\Nhidden] {Output layer};

    \path[dashed] (H2-0) edge (O); %<-- added %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{tikzpicture}
% End of code
\end{document}

With semic automatic placement of nodes

Loops and polar coordinates:

\documentclass[border=1cm]{standalone}

\usepackage{tikz}
\usepackage{xcolor}

\begin{document}
 \begin{tikzpicture}
  \foreach \x /\alph/\name in {0/a/one, 60/b/two, 120/c/three, 180/d/four, 240/e/five, 300/f/six}{
  \node[circle, fill=green,minimum width=15mm,draw,shading=axis,top color=green, bottom color=green!50!black] (\alph) at (\x:3cm) {\name}; }

  \foreach \alpha in {a,b,c,d,e,f}%
  {%
  \foreach \alphb in {a,b,c,d,e}%
  {%
   \draw (\alpha) -- (\alphb);%
  }}

 \end{tikzpicture}


\end{document}

enter image description here

Complete manual placement

Improvement: using the backgound library. This makes it possible to draw the nodes first and then the lines, without seeing the lines crossing the nodes.

\documentclass[border=1cm]{standalone}

\usepackage{tikz}
\usetikzlibrary{backgrounds}

\usepackage{xcolor}


\begin{document}
 \begin{tikzpicture}[every node/.style={circle, fill=green,minimum width=15mm,draw,shading=axis,top color=green, bottom color=green!50!black}]

 \node (a) at (0,0){ one};
 \node (b) at (0,2) {two};
 \node (c) at (4,2) {three}; 
 \node (d) at (4,0) {four}; 
 \node (e) at (8,-2) {five}; 
 \node (f) at (8,3){six}; 


\begin{scope}[on background layer]
  \foreach \alpha in {a,b,c,d,e,f}%
  {%
  \foreach \alphb in {a,b,c,d,e}%
  {%
   \draw (\alpha) -- (\alphb);%
  }}
\end{scope}
 \end{tikzpicture}


\end{document}

enter image description here

tkz-berge solution

I'm not quite a familiar with tkz-berge but this should do:

\documentclass[border=5cm]{standalone}

\usepackage{tikz}    
\usepackage{xcolor}
\usepackage{tkz-berge}

\begin{document}

\begin{tikzpicture} 
    \renewcommand*{\VertexBallColor}{green!80!black}
    \GraphInit[vstyle=Shade] 
     \grComplete[RA=5]{6}
\end{tikzpicture}

\begin{tikzpicture}
    \renewcommand*{\VertexBallColor}{green!80!black}
    \GraphInit[vstyle=Shade] 

    \grPath[Math,prefix=p,RA=2,RS=0]{2} 
    \grPath[Math,prefix=q,RA=2,RS=3]{2}

    \begin{scope}[xshift=4cm, yshift=-1cm]
        \grPath[Math,prefix=r,RA=2,RS=0]{1} 
    \end{scope}

    \begin{scope}[xshift=4cm, yshift=4cm]
        \grPath[Math,prefix=s,RA=2,RS=0]{1} 
    \end{scope}

    \foreach \from in { 0,...,1}{
        \EdgeFromOneToAll{p}{q}{\from}{2}
    }
    \EdgeFromOneToAll{r}{q}{0}{2}
    \EdgeFromOneToAll{r}{p}{0}{2}

    \EdgeFromOneToAll{s}{q}{0}{2}
    \EdgeFromOneToAll{s}{p}{0}{2}
    \EdgeFromOneToAll{s}{r}{0}{1}

\end{tikzpicture}

\end{document}

The first plot is quite easy with the package but trying manual placemant is rather typing intense. This package is not made for such special stuff, since there is no build in command for having one path of vertices but with complete manual placement. (At least as far as i know.). So i'd like to present a proper automatic (One) and more a hackish possibility (Two) which provides near to manual placement

enter image description here

[Tex/LaTex] Drawing neural network with tikz

Here, we a have a festival of \foreach:

\documentclass[border=0.125cm]{standalone}
\usepackage{tikz}
\usetikzlibrary{positioning}
\begin{document}

\tikzset{%
  every neuron/.style={
    circle,
    draw,
    minimum size=1cm
  },
  neuron missing/.style={
    draw=none, 
    scale=4,
    text height=0.333cm,
    execute at begin node=\color{black}$\vdots$
  },
}

\begin{tikzpicture}[x=1.5cm, y=1.5cm, >=stealth]

\foreach \m/\l [count=\y] in {1,2,3,missing,4}
  \node [every neuron/.try, neuron \m/.try] (input-\m) at (0,2.5-\y) {};

\foreach \m [count=\y] in {1,missing,2}
  \node [every neuron/.try, neuron \m/.try ] (hidden-\m) at (2,2-\y*1.25) {};

\foreach \m [count=\y] in {1,missing,2}
  \node [every neuron/.try, neuron \m/.try ] (output-\m) at (4,1.5-\y) {};

\foreach \l [count=\i] in {1,2,3,n}
  \draw [<-] (input-\i) -- ++(-1,0)
    node [above, midway] {$I_\l$};

\foreach \l [count=\i] in {1,n}
  \node [above] at (hidden-\i.north) {$H_\l$};

\foreach \l [count=\i] in {1,n}
  \draw [->] (output-\i) -- ++(1,0)
    node [above, midway] {$O_\l$};

\foreach \i in {1,...,4}
  \foreach \j in {1,...,2}
    \draw [->] (input-\i) -- (hidden-\j);

\foreach \i in {1,...,2}
  \foreach \j in {1,...,2}
    \draw [->] (hidden-\i) -- (output-\j);

\foreach \l [count=\x from 0] in {Input, Hidden, Ouput}
  \node [align=center, above] at (\x*2,2) {\l \\ layer};

\end{tikzpicture}

\end{document}

Although it seems unwise to have n denote the number of nodes in each layer when they could be different and the arrangement of the diagram suggests they are not.

enter image description here

Best Answer

Related Solutions

[Tex/LaTex] Fully connected network diagram

With semic automatic placement of nodes

Complete manual placement

tkz-berge solution

[Tex/LaTex] Drawing neural network with tikz

Related Question