[Tex/LaTex] I need pointers on how to draw the neural-network diagram

Question

Please help me draw this diagram. I'm practicing on TikZ, but it's so difficult.

I don't know which library I should use, matrix/chain or another something ?
I also don't know how to draw a circle connector like picture.
Please help me draw this picture.

Answer 1

One option using TikZ; the only point that perhaps deserves some comment is the curved arrow, produced using an arc path and a decoration:

\documentclass{article}
\usepackage{amsmath}
\usepackage{tikz}
\usetikzlibrary{positioning,decorations.markings,calc}

\begin{document}

\begin{tikzpicture}[
node distance=2cm and 1cm,
mynode/.style={
  draw,
  text width=6cm,
  align=center,
  text height=3ex,
  text depth=1.5ex,
}
]
\node[mynode] (out) {Output};
\node[mynode,text width=4cm,below=of out] (sh) {State/Hidden};
\node[mynode,below=of sh] (in) {Input};

\draw[-latex] (in) -- node[fill=white] {Weights $V$}(sh);
\draw[-latex] (sh) -- node[fill=white] {Weights $W$}(out);

\begin{scope}[radius=1.2cm]
\draw[
decoration={
  markings,
  mark=at position 0.999 with {\arrow{latex}}
  },
postaction=decorate
]
(sh.16) arc[start angle=160,end angle=-150] (sh.-17);

\node[xshift=1.3cm,anchor=west,text width=2.8cm,align=center] 
at (sh.east) {Weights $U$\\(dealyed)};
\end{scope}

\node[anchor=west,text width=4cm] at ([xshift=4cm] $ (in.north)!0.5!(sh.south)$ ) {%
        $\displaystyle
        \begin{aligned}
            &y_j(t)=f(\mathrm{net}_j)\\
            &\mathrm{net}_j(t)=\sum_i v_{ji} x_i(t)+\sum_h u_{jh}y_h(t-1)+\theta_j
        \end{aligned}$
};

\node[anchor=west,text width=4cm] at ([xshift=4cm] $ (sh.north)!0.5!(out.south)$ ) {%
        $\displaystyle
        \begin{aligned}
            &y_k(t)=g(\mathrm{net}_k)\\
            &\mathrm{net}_k(t)=\sum_j w_{kj} y_j(t)+\theta_k
        \end{aligned}$
};
\end{tikzpicture}

\end{document}