[Tex/LaTex] Softmax activation function

plottikz-pgf

I would like to plot the softmax activiation function. I know that there is already a similar question but unfortunately neither the comments, nor the answer provided there could really help me so far. This is my current code:

\begin{figure}[t]
    \begin{tikzpicture}
        \begin{axis}[
                    ylabel=$\sigma(z)_j$,
                    xlabel=$z$,
                    xmin=-5,
                    xmax=5]
           \addplot[blue,domain=-5:5,samples=51] 
           {exp(x)/sumexp(x,-4,0)};
        \end{axis}
    \end{tikzpicture}
  \caption{Softmax activation function.}
 \end{figure}

Attempting to compile this code always results in the following error message:

Package PGF Math Error: Unknown function `sumexp' (in
'exp(x)/sumexp(x,-4,0)'). \end{tikzpicture} }

Any ideas are highly appreciated.

//edit:

After adding the following definition (that I foolishly forgot to add before):

\pgfmathdeclarefunction{sumexp}{3}{%
\begingroup%
\pgfkeys{/pgf/fpu,/pgf/fpu/output format=fixed}%
\pgfmathsetmacro{\myx}{#1}%
\pgfmathtruncatemacro{\myxmin}{#2}%
\pgfmathtruncatemacro{\myxmax}{#3}%
\pgfmathsetmacro{\mysum}{0}%
\pgfplotsforeachungrouped\XX in {\myxmin,...,\myxmax}%
{\pgfmathsetmacro{\mysum}{\mysum+exp(\XX)}}%
\pgfmathparse{\mysum+exp(#1)}%
\pgfmathsmuggle\pgfmathresult\endgroup%
}%

I end up with the following error message:

Package PGF Math Error: Sorry, an internal routine of the floating
point unitt was near '-4.0000000000'. (in 'exp(x)/sumexp(x,-4,0)').
\end{tikzpicture} }

Best Answer

Here the (equivalent) effect of /pgf/fpu/output format=fixed is restricted to the very end of function sumexp only.

\documentclass{article}
\usepackage{pgfplots}
\pgfplotsset{compat=1.18}

\makeatletter
\pgfmathdeclarefunction{sumexp}{3}{%
  \begingroup%
  \pgfkeys{/pgf/fpu}% "/pgf/fpu/output format=fixed" removed
  \pgfmathsetmacro{\myx}{#1}%
  \pgfmathtruncatemacro{\myxmin}{#2}%
  \pgfmathtruncatemacro{\myxmax}{#3}%
  \pgfmathsetmacro{\mysum}{0}%
  \pgfplotsforeachungrouped\XX in {\myxmin,...,\myxmax}%
    {\pgfmathsetmacro{\mysum}{\mysum+exp(\XX)}}%
  \pgfmathparse{\mysum+exp(#1)}%
  \pgfmathfloattofixed\pgfmathresult%  added
  \pgfmathsmuggle\pgfmathresult\endgroup%
}%
\makeatother

\begin{document}
\begin{tikzpicture}
    \begin{axis}[
                ylabel=$\sigma(z)_j$,
                xlabel=$z$,
                xmin=-5,
                xmax=5]
       \addplot[blue,domain=-5:5,samples=51] 
       {exp(x)/sumexp(x,-4,0)};
    \end{axis}
\end{tikzpicture}
\end{document}

enter image description here

Please help me check if the plot looks alright since I hardly know anything about the softmax function.

Some immature words:

It is pgfplots commit d2fbb2a that led to the error

! Package PGF Math Error: Sorry, an internal routine of the floating point unit got an ill-formatted floating point number `-4.0000000000'. The unreadable part was near ''..

It seems that with /pgf/fpu=true, /pgf/fpu/output format=fixed doesn't work with \pgfplotsforeachungrouped, since the latter one may call \pgfmathsubtract@ (in \pgfplotsforeachungroupeduniform@) which, when fpu is activated, always expects input in float format.

Here I employed the possibly easiest change: restrict the scope of /pgf/fpu/output format=fixed. In a bigger picture, perhaps the \pgfplotsforeachungrouped or \pgfmathsubtract@ should be enhanced.

Related Question