MATLAB: Long Short Term Memory

deep learninglstmrnn

Dear all, I am trying to implement a LSTM, for sequence-to-label classification duty. Since I know all the sequence, I am using BILSTM. My training dataset is composed by 12000 observations, of lenght 2048, with 2 features. Such dataset is stored in a cell array, having dimension 12000×1, where each cell is 2×2048, and binary label (0 or 1) in a categorigal array. The architecture used for this aim is the follow:
inputSize = 2;
numHiddenUnits1 = 200;
numHiddenUnits2 = 150;
numClasses = 2;
layers = [ …
    sequenceInputLayer(inputSize)
    bilstmLayer(numHiddenUnits1,'OutputMode','sequence')
    bilstmLayer(numHiddenUnits2,'OutputMode','last')
    fullyConnectedLayer(numClasses)
    softmaxLayer
    classificationLayer];
maxEpochs = 30;
miniBatchSize = 25;
options2 = […
 trainingOptions('adam', ...
    'ExecutionEnvironment','gpu', ...
    'GradientThreshold',1, ...
    'MaxEpochs',maxEpochs, ...
    'MiniBatchSize',miniBatchSize, ...
    'SequenceLength','longest', ...
    'Shuffle','never', ...
    'Verbose',1, ...
    'Plots','training-progress',...
    'CheckpointPath','C:\Users\jwb15214\Desktop\CNN_MATLABtool\CV-CNN monodimensional signal\CV-CNN-master\CV-CNN\CheckPointsPath');
net5 = trainNetwork(train_data_cell,categorical_label_new,layers,options2);
The way how LSTM is explained on the Matlab help, let me understand that each LSTM unit is connected to a sample of the input sequence. In my case, I choose to set the first LSTMLayer a number of hidden layer equal to 200, but with a sequence length of 2048. How Does it work in this case? Is there any documentation explaing the correlation between input and output of a bilstm? What is the difference between the 'sequence' mode and the 'last' mode in terms of filter size and features map?
Kind regards Alessio

Best Answer

By my understanding you want some detail information related to LSTM. For more detail information related to LSTM, you can find it in the attached documentation link attached:
https://www.mathworks.com/discovery/lstm.html
https://colah.github.io/posts/2015-08-Understanding-LSTMs/
I think the above links will resolve your first two questions. To create an LSTM network for sequence-to-label classification, create a layer array containing a sequence input layer, an LSTM layer, a fully connected layer, a softmax layer, and a classification output layer. Specify the size of the sequence input layer to be the number of features of the input data. Specify the size of the fully connected layer to be the number of classes. You do not need to specify the sequence length. For the LSTM layer, specify the number of hidden units and the output mode 'last'. To create an LSTM network for sequence-to-sequence classification, use the same architecture for sequence-to-label classification, but set the output mode of the LSTM layer to 'sequence'. Here is the documentation link:
https://www.mathworks.com/help/deeplearning/ug/long-short-term-memory-networks.html#mw_9cf7b3dc-bc5e-40f2-a20a-82bdd6df1b03
Related Question