MATLAB: Error while trying to train network using multiple cpus

Image Processing ToolboxMATLABneural network

Hi, I'm following the super resolution image neural network example from the mathwork website. Everything works fine except utilization of multiple cpus. My GPU is old. So, I've given up gpu option. But when I try to use multiple cpus by putting following options in the 'trainNetwork' I get errors.

,...
'ExecutionEnvironment', 'parallel');

Errors are as follows:

Starting parallel pool (parpool) using the 'local' profile ...
connected to 8 workers.
Error using trainNetwork (line 154)
MiniBatchable Datastore is not distributable.
Error in trainVDSR (line 75)
    net = trainNetwork(source,layers,options);
Caused by:
    Error using nnet.internal.cnn.DataDispatcherFactory.createDataDispatcher (line 100)
    MiniBatchable Datastore is not distributable.

Any idea why it's complaining?

Thank you, C

Best Answer

The error means that the vdsrImagePatchDatastore does not support parallel training. This is a custom datastore created for this example. You could try adding the Distributable interface yourself to get it working.

I'll admit the error message is confusing and we should do something about it.

Related Solutions

MATLAB: How to change color model of all pictures in image datastore

I hope I've figured this problem with Transform datastore:

First error

Error using trainNetwork (line 170)
Invalid transform function defined on datastore.
Error in transferlearning (line 37)
CNN = trainNetwork(dsnew.train, lgraph, opts);
Caused by:
    Error using nnet.internal.cnn.util.NetworkDataValidator/assertDatastoreHasResponses (line 140)
    Invalid transform function defined on datastore.
        Error using rgb2hsv>parseInputs (line 95)
        MAP must be a Mx3 array.

was figured out by using im2uint8 function.

im2uint8(rgb2hsv(dataIn));

Second error

Error using trainNetwork (line 170)
Invalid training data. Responses must be nonempty.

was figured out thanks to this script in transformFcn file:

function dataOut = transformFcn(dataIn)
B = table2cell(dataIn);
NoImg = size(B,1);
for i=1:NoImg
    B{i,1} = im2uint8(rgb2hsv(B{i,1}));
end
dataOut = cell2table(B);
end

In main file:

imds.train = imageDatastore(fullfile(trainDirectoryName, categories), 'LabelSource', 'foldernames');
imds.train = splitEachLabel(imds.train,5, 'randomize');
auimds.train = augmentedImageDatastore(sizeImg, imds.train);
dsnew.train = transform(auimds.train, @transformFcn);
dsnew.train.UnderlyingDatastore.MiniBatchSize = 10;

ReadFcn takes only one image, instead Transform datastore takes whole batch of images (size of this batch is defined in the last line: dsnew.train.UnderlyingDatastore.MiniBatchSize; default is 128).

Hope this helps :-)

MATLAB: No appropriate method, property, or field ‘Files’ for class ‘augmentedImageDatastore’.

Hack way:

old_warning_state = warning('off', 'MATLAB:structOnObject');
temp = struct(augimdsValidation);
warning(old_warning_state);
valFrequency = floor(numel(temp.DatastoreInternal.Files)/miniBatchSize);

Best Answer

Related Solutions

MATLAB: How to change color model of all pictures in image datastore

MATLAB: No appropriate method, property, or field ‘Files’ for class ‘augmented​ImageDatas​tore’.

Related Question

MATLAB: No appropriate method, property, or field ‘Files’ for class ‘augmentedImageDatastore’.