MATLAB: Neural network validation accuracy on Test. Images

image processingimage processing performancevalidation testing

Hi All Professionals,
I have this code works fine!!
What I am trying to observe is the performance of the test images, can some direct me to a link or an example to see this process visually?
How is validation done on convolution networks after the training?
How can I see the validation performance and visual how accurate the network was at this phase?
Can someone assist me please?
thank you in advance have a great day!
%% Turen Of PNG Warnings
[~, warnid] = lastwarn; %get identifier of warning
warning('off', warnid); %turn warning off
% clc
% close all
% clear 
%% Training The R-CNN Detector On Gun Dataset
%% Step 1 Loading Training Data & Network Layers.
load('ReviewSim264.mat');
load('layers40.mat');
save new.mat ReviewSim layers;
load('new.mat', 'ReviewSim','layers');
summary(ReviewSim);
%% Step 2 Specifing Image Location
imDir = fullfile(matlabroot,'Revims');
addpath(imDir);
%% Step 3 Accessing Content of Folder TrainingSet Using Datastore
imds =imageDatastore(imDir,'IncludeSubFolders',true,'LabelSource','Foldernames');
tbl = countEachLabel(imds);
%imds.Labels
% 
%% Step 4 Splitting Inputs Into Training and Testing Sets
[imdsTrain,imdsValidation] = splitEachLabel(imds,0.7,'randomized');
size(imdsTrain);
%% Step 5 Specifying Input Size OF 1st Network Layer
inputSize = layers(1).InputSize;
labelData = ReviewSim.imageFilename;
%% Step 6 Replacing Final Layer/Last 3 Configure For Classes
% Finetuning these 3 layers for new classification
% Extracting all Layers except the last 3
layersTransfer = layers(1:end-3);
% Stipulating Amount Of Classes 
numClasses = numel(categories(imdsTrain.Labels));
% Adding Newly Edited Layers
Tlayers = [layersTransfer
  fullyConnectedLayer(numClasses,'Name','fullyConn')
  softmaxLayer('Name','softmax')
  classificationLayer('Name','classoutput','Classes', 'auto')];
%% Step 7 Warping Images For Added Accuracy
pixelRange = [-30 30];
imageAugmenter = imageDataAugmenter(...
'RandRotation',[-40 40],...
'RandXReflection',true,...
'RandYReflection',true,...
'RandXShear',[-15 15],...
'RandYShear',[-10 10],...
'RandXTranslation',pixelRange, ... 
'RandYTranslation',pixelRange);
%% Step 8 Deploying Augmentation Preventing Overfitting   
augimdsValidation = augmentedImageDatastore(inputSize,imdsValidation,...
'ColorPreprocessing','gray2rgb','DataAugmentation',imageAugmenter);
augmentedTrainingSet = augmentedImageDatastore(inputSize,imdsTrain,...
'ColorPreprocessing', 'gray2rgb','DataAugmentation',imageAugmenter);
%% Step 9 Specifying Option Features
% automatically drop the learn rate during training using a piecewise
% learn rate schedule
options = trainingOptions('sgdm',... 
    'Momentum',0.8,... 
    'InitialLearnRate', 1e-3,... 
    'LearnRateSchedule', 'piecewise', ... 
    'LearnRateDropFactor', 0.1, ... 
    'Shuffle','every-epoch', ... 
    'LearnRateDropPeriod', 14, ... 
    'L2Regularization', 1e-4, ... 
    'MaxEpochs',8,... 
    'MiniBatchSize',20,...
    'Verbose', true);
%% Step 10 Combining All Network Variables For Training Sequence
netTransfer = trainNetwork(augmentedTrainingSet,Tlayers,options);
%% Step 11 Training The R-CNN Detector/Display Network Layers 
rcnn = trainRCNNObjectDetector(ReviewSim, netTransfer, options, 'NegativeOverlapRange', [0 0.3]);
rcnn.RegionProposalFcn;
network = rcnn.Network;
layers = network.Layers;
%% Step 12 Displaying RCNN Class Names
rcnn.ClassNames;
%% Step 13 Displaying Strongest Detection Result.
img = imread('3.jpg');
%loop through all images of the augmented dataset and predict guns
%locations in them
%compare the output of the rcnn with the truth location of the image guns.
[bbox, score, label] = detect(rcnn, img, 'MiniBatchSize', 8,'SelectStrongest',true);
[score, idx] = max(score);
bbox = bbox(idx, :); 
annotation = sprintf('%s: (Confidence = %f)', label(idx), score);
detectedImg = insertObjectAnnotation(img, 'rectangle', bbox, annotation);
figure
imshow(detectedImg);
%test the network over the unknown validation dataset

Best Answer

trainRCNNObjectDetector doesn’t support “ValidationData”, “ValidationFrequency” and “ValidationPatience” options as of now. Hence, it is not possible to directly feed the validation data to the trainingOptions parameter.
Possible workaround for the k-fold validation could be to run a loop “k” times.
  • In each loop, split the train set randomly into two sets, where one set will be used for training and the other for validation. You can use crossvalind or cvpartition functions for this purpose.
  • Use this new train set for training the model and after the training is done, use the validation set to evaluate the model.
This above step is done “k” times.
For more information of k-fold cross validation, you can refer to the following links:
1) https://www.mathworks.com/help/bioinfo/ref/crossvalind.html
2) https://www.mathworks.com/help/stats/cvpartition.html
Related Question