[Tex/LaTex] Plot gradient descent

3dpgfplots

How can I plot the gradient descent as a 3d graph in LaTeX? It should look something like this, but it can also look a lot more simple, like this.

I'm pretty new to LaTeX and I don't really know where to start, so I'd really appreciate some help.

Thank you

Best Answer

This is at least a start. You can define function that compute the components of the gradient numerically for a given function. Then you do a loop to produce the next coordinate from the previous one and the gradient at the previous coordinate. Many variations are possible, as usual (and I hope that this does not not lead to many comments requesting to spell out these variations ;-).

\documentclass[tikz,border=3mm]{standalone}
\usetikzlibrary{decorations.pathreplacing}
\tikzset{arrowed/.style={decorate,
decoration={show path construction, 
moveto code={},
lineto code={
\draw[#1] (\tikzinputsegmentfirst) --  (\tikzinputsegmentlast);
},
curveto code={},
closepath code={},
}},arrowed/.default={-stealth}}
\usepackage{pgfplots}
\pgfplotsset{gradient function/.initial=f,
dx/.initial=0.01,dy/.initial=0.01}
\pgfmathdeclarefunction{xgrad}{2}{%
\begingroup%
\pgfkeys{/pgf/fpu,/pgf/fpu/output format=fixed}%
\edef\myfun{\pgfkeysvalueof{/pgfplots/gradient function}}%
\pgfmathparse{(\myfun(#1+\pgfkeysvalueof{/pgfplots/dx},#2)%
-\myfun(#1,#2))/\pgfkeysvalueof{/pgfplots/dx}}%
 % \pgfmathsetmacro{\mysum}{\mysum+\myfun(\value{isum},#2)}%
\pgfmathsmuggle\pgfmathresult\endgroup%
}%
\pgfmathdeclarefunction{ygrad}{2}{%
\begingroup%
\pgfkeys{/pgf/fpu,/pgf/fpu/output format=fixed}%
\edef\myfun{\pgfkeysvalueof{/pgfplots/gradient function}}%
\pgfmathparse{(\myfun(#1,#2+\pgfkeysvalueof{/pgfplots/dy})%
-\myfun(#1,#2))/\pgfkeysvalueof{/pgfplots/dy}}%
 % \pgfmathsetmacro{\mysum}{\mysum+\myfun(\value{isum},#2)}%
\pgfmathsmuggle\pgfmathresult\endgroup%
}%

\pgfplotsset{compat=1.17}
\begin{document}
\begin{tikzpicture}
\begin{axis}[width=12cm,%
    declare function={f(\x,\y)=cos(deg(\x)*0.8)*cos(deg(\y)*0.6)*exp(0.1*\x);}]
 \addplot3[surf,shader=interp,domain=-4:4,%samples=81
 ]{f(x,y)};
 \edef\myx{0.15} % first x coordinate
 \edef\myy{-0.15} % first y coordinate
 \edef\mystep{-2}% negative values mean descending
 \pgfmathsetmacro{\myf}{f(\myx,\myy)}
 \edef\lstCoords{(\myx,\myy,\myf)}
 \pgfplotsforeachungrouped\X in{0,...,5}
 {
 \pgfmathsetmacro{\myx}{\myx+\mystep*xgrad(\myx,\myy)}
 \pgfmathsetmacro{\myy}{\myy+\mystep*ygrad(\myx,\myy)}
 \pgfmathsetmacro{\myf}{f(\myx,\myy)}
 \edef\lstCoords{\lstCoords\space (\myx,\myy,\myf)}
 }
 \addplot3[samples y=0,arrowed] coordinates \lstCoords;
\end{axis}
\end{tikzpicture}
\end{document}

A perhaps more useful variation is to normalize the steps.

\documentclass[tikz,border=3mm]{standalone}
\usetikzlibrary{decorations.pathreplacing}
\tikzset{arrowed/.style={decorate,
decoration={show path construction, 
moveto code={},
lineto code={
\draw[#1] (\tikzinputsegmentfirst) --  (\tikzinputsegmentlast);
},
curveto code={},
closepath code={},
}},arrowed/.default={-stealth}}
\usepackage{pgfplots}
\pgfplotsset{gradient function/.initial=f,
dx/.initial=0.01,dy/.initial=0.01}
\pgfmathdeclarefunction{xgrad}{2}{%
\begingroup%
\pgfkeys{/pgf/fpu,/pgf/fpu/output format=fixed}%
\edef\myfun{\pgfkeysvalueof{/pgfplots/gradient function}}%
\pgfmathparse{(\myfun(#1+\pgfkeysvalueof{/pgfplots/dx},#2)%
-\myfun(#1,#2))/\pgfkeysvalueof{/pgfplots/dx}}%
 % \pgfmathsetmacro{\mysum}{\mysum+\myfun(\value{isum},#2)}%
\pgfmathsmuggle\pgfmathresult\endgroup%
}%
\pgfmathdeclarefunction{ygrad}{2}{%
\begingroup%
\pgfkeys{/pgf/fpu,/pgf/fpu/output format=fixed}%
\edef\myfun{\pgfkeysvalueof{/pgfplots/gradient function}}%
\pgfmathparse{(\myfun(#1,#2+\pgfkeysvalueof{/pgfplots/dy})%
-\myfun(#1,#2))/\pgfkeysvalueof{/pgfplots/dy}}%
 % \pgfmathsetmacro{\mysum}{\mysum+\myfun(\value{isum},#2)}%
\pgfmathsmuggle\pgfmathresult\endgroup%
}%

\pgfplotsset{compat=1.17}
\begin{document}
\begin{tikzpicture}
\begin{axis}[width=12cm,%
    declare function={f(\x,\y)=cos(deg(\x)*0.8)*cos(deg(\y)*0.6)*exp(0.1*\x);}]
 \addplot3[surf,shader=interp,domain=-4:3,%samples=81
 ]{f(x,y)};
 \edef\myx{1} % first x coordinate
 \edef\myy{0.25} % first y coordinate
 \edef\mystep{-0.25}% negative values mean descending
 \pgfmathsetmacro{\myf}{f(\myx,\myy)}
 \edef\lstCoords{(\myx,\myy,\myf)}
 \pgfplotsforeachungrouped\X in{0,...,5}
 {
 \pgfmathsetmacro{\mydx}{xgrad(\myx,\myy)}
 \pgfmathsetmacro{\mydy}{ygrad(\myx,\myy)}
 \pgfmathsetmacro{\myscale}{\mystep/sqrt(\mydx*\mydx+\mydy*\mydy)}
 \pgfmathsetmacro{\myx}{\myx+\myscale*\mydx}
 \pgfmathsetmacro{\myy}{\myy+\myscale*\mydy} 
 \pgfmathsetmacro{\myf}{f(\myx,\myy)}
 \edef\lstCoords{\lstCoords\space (\myx,\myy,\myf)}
 }
 \addplot3[samples y=0,arrowed] coordinates \lstCoords;
\end{axis}
\end{tikzpicture}
\end{document}

One may also use a quiver plot.

Related Solutions

PGFPlots – Plotting Third Variable as Color Gradient

Adding to your answer: You can "fake" the reversed colorbar by rotating it and putting the labels on the other side. A colorbar is essentially another axis, so all the styles you get with normal axes can be used.

\documentclass{article}
\usepackage{pgfplots}

\begin{document}

\begin{tikzpicture}[scale=0.75]
\begin{axis}[colorbar,colormap/cool,
  colorbar style={xshift=1cm,
    yticklabel pos=left,
    yticklabel style={anchor=west},
    rotate=180}]
\addplot[scatter,point meta=explicit,ultra thick,point meta min={1}, point meta max={700}]
    table[x=S,y=POT_T, meta=DB, col sep=semicolon]{
      S;POT_T;DB
      0;0;650
      1;0.5;400
      2;2;150
};
\end{axis}
\end{tikzpicture}

\end{document}

axis with pseudo-reversed colorbar

[Tex/LaTex] Create 3D heatmap density plot using TikZ

As Jake said in the comments, this is probably beyond the realm of writing directly into TikZ, but you could probably still get it done, if you wanted some other benefits of TikZ. (e.g., fonts that are consistent with the rest of the documents)

If you already have some software to generate 3D plots separately from LaTeX (such as MATLAB or Mathematica), then you might want to look to see if there’s a library that translates drawing code from that software into TikZ that you can embed in LaTeX. Here are some examples of what I mean:

matlab2tikz
tikzDevice (R, described below)
GNUPLOT + TikZ

Doing it externally and importing TikZ is almost certainly the way to go. It will be a lot easier than trying to get it to get something that’s workable, and the end result will be a lot better.

Below I’ve given a brief example of how I do it, using R. It’s unlikely that you’ll be able to use my solution directly, but it might give you an idea of what I mean, and perhaps give you a pointer in the right direction.

A personal example:

I often use R to do my plots (R is a free, statistical programming language), and R has a useful module for exporting to TikZ. This TikZ code can then be loaded into a LaTeX document and compiled as normal.

The R package for 3d contour plots is contour3d, and here’s an example from that documentation page:

require(tikzDevice)
require("misc3d")

tikz("isosurface.tex", width=5, height=5)

##################################################################
#                                                                #
# Example from the R documentation                               #
# http://rss.acs.unt.edu/Rdoc/library/misc3d/html/contour3d.html #
#                                                                #
##################################################################

nmix3 <- function(x, y, z, m, s) {
      0.4 * dnorm(x, m, s) * dnorm(y, m, s) * dnorm(z, m, s) +
      0.3 * dnorm(x, -m, s) * dnorm(y, -m, s) * dnorm(z, -m, s) +
      0.3 * dnorm(x, m, s) * dnorm(y, -1.5 * m, s) * dnorm(z, m, s)
  }
  f <- function(x,y,z) nmix3(x,y,z,.5,.5)
  g <- function(n = 40, k = 5, alo = 0.1, ahi = 0.5, cmap = heat.colors) {
      th <- seq(0.05, 0.2, len = k)
      col <- rev(cmap(length(th)))
      al <- seq(alo, ahi, len = length(th))
      x <- seq(-2, 2, len=n)
      contour3d(f,th,x,x,x,color=col,alpha=al)
      rgl.bg(col="white")
  }
  g(40,5)
  gs <- function(n = 40, k = 5, cmap = heat.colors, ...) {
      th <- seq(0.05, 0.2, len = k)
      col <- rev(cmap(length(th)))
      x <- seq(-2, 2, len=n)
      m <- function(x,y,z) x > .25 | y < -.3
      contour3d(f,th,x,x,x,color=col, mask = m, engine = "standard",
                scale = FALSE, ...)
      rgl.bg(col="white")
  }
  gs(40, 5, screen=list(z = 130, x = -80), color2 = "lightgray", cmap=rainbow)

### End of example

dev.off()

Most of that is just drawing the 3D plot, but the lines at the top and bottom turn the plotted output into TikZ, which gets saved to isosurface.tex.

Once I've run that script, I can \input that document into another LaTeX file. For example:

\documentclass{article}

\usepackage{tikz}

\begin{document}

\input{isosurface.tex}

\end{document}

This is what it looks like:

enter image description here

and the rendering in the LaTeX-ed PDF is remarkably faithful to the original:

enter image description here

Some caveats:

For a 3D plot, the associated TikZ file is ~big. Approximately 2.2 MB, which is an order of magnitude bigger than most TikZ documents.
It will slow you down. That smallish example added ~30 seconds to the compile time. If you have more complicated ones, or lots of them, expect to get significantly slowed down. You might want to look at using something like TikZ’s externalize library to avoid the worst performance hit.

If you want to use R, then questions about how to draw your plots are off-topic at this Stack Exchange, but you might want to head over to Cross Validated if you need a hand. Topics about R and data visualisation are more on-topic there.

Best Answer

Related Solutions

PGFPlots – Plotting Third Variable as Color Gradient

[Tex/LaTex] Create 3D heatmap density plot using TikZ

Related Question