MATLAB: Shallow Neural Network with Stochastic Gradient Descent

Question

Hi

I am looking to design a shallow neural network with the gradient descent algorithm. From what I can find from research and Googling, the old Neural Networks toolbox didn't have this option. I understand the options I am looking for are available with the CNN functions. But I just need a simple multilayered Aritificial Neural Network with stochastic gradient descent. I have the neural network setup with feedforwardnet but I can't find any function for stochastic gradient descent as a training function, i.e. no mini-Batches.

So far, I have the simple gradient descent: net.trainFcn = 'traingd';

Am I missing something? Or coding it from scratch is the only option?

Answer 1

The following command lists all the training functions available:

help nntrain

From the output I can see that stochastic gradient descent is not there in case of shallow neural networks.

Since that you want a simple Multilayered ANN, refer to List of Deep Learning Layers & use appropriate layer from Input Layers followed by fullyconnectedLayers & appropriate Output Layer. Then use the 'sgdm' in trainingOptions.

Also refer to Deep Learning Toolbox — Examples