MATLAB: Size of Mini-Batch in deep learning

audiodeep learningmachine learningMATLAB

Hi,

I am trying to train some deep learning models and trying to set up good value for the Mini-Batch. As you may know, the value of the Mini-Batch should be between 1 and M where M is the size of the batch.

My question is, once I extracted some training features and prepare the data for training. How can I tell the size of my batch so I can set the Mini-Batch?

Thanks

Best Answer

As per the docs, the "mini-batch is a subset of the training set that is used to evaluate the gradient of the loss function and update the weights". So the batch size would be the entire training set.

Related Solutions

MATLAB: Monitor Deep Learning Training Progress

I tried reproducing the above issue using an example provided in the documentation -

digitDatasetPath = fullfile(matlabroot,'toolbox','nnet','nndemos','nndatasets','DigitDataset');
imds = imageDatastore(digitDatasetPath, 'IncludeSubfolders',true,'LabelSource','foldernames');
numTrainFiles = 750;
[imdsTrain,imdsValidation] = splitEachLabel(imds,numTrainFiles,'randomize');
inputSize = [28 28 1];
numClasses = 10;
layers = [
    imageInputLayer(inputSize)
    convolution2dLayer(5,20)
    batchNormalizationLayer
    reluLayer
    fullyConnectedLayer(numClasses)
    softmaxLayer
    classificationLayer];
options = trainingOptions('sgdm', ...
    'MaxEpochs',4, ...
    'ValidationData',imdsValidation, ...
    'ValidationFrequency',30, ...
    'Verbose',false, ...
    'Plots','training-progress');
net = trainNetwork(imdsTrain,layers,options);

I was able to view the training progress

Please crosscheck the spellings of Name Value pairs in trainingOptions

MATLAB: Matlab trainNetwork mini-batch size

If I understand correctly, you expect the 'miniBatchSize' to create the batches along the time dimension. But this is not what 'miniBatchSize' does. 'MiniBatchSize' creates batches in the observation dimension, not the time dimension. To create batches in the time dimension, you can use the 'SequenceLength' parameter.The default 'SequenceLength' parameter is ‘longest’, which means that the batch sequence length is determined by the longest sequence in the batch.

Best Answer

Related Solutions

MATLAB: Monitor Deep Learning Training Progress

MATLAB: Matlab trainNetwork mini-batch size

Related Question