MATLAB: What happens when a parallel worker finishes early

parallel computingParallel Computing Toolboxparfor

Hi everyone,

I have a benchmark code running where I test the time it takes from start to finish. I noticed that the workers are not synchronized. What will happen when one worker finishes early? This is a parfor loop of 20 workers on a 20-core server.

Good day!

Best Answer

When MATLAB runs a parfor loop, it attempts to split the iterations of the loop across the workers to keep them all busy simultaneously. However, this is not always possible (especially if you have only a very small number of loop iterations, and the iterations take different amounts of time). So, towards the end of the parfor loop, some of the workers will be idle waiting for the last workers to finish.

Related Solutions

MATLAB: Is the communicating job stuck behind a concurrent job in queue in R2017b

The resolution is to avoid starting a parallel pool when submitting other jobs to the cluster.

The "parpool" object is generally used for interactive "parfor" or "smpd" operations. When the parallel pool is created with "parpool", it creates a concurrent job on the cluster that readies all of the workers to be available for when the user interactively calls "parfor" or "smpd". The concurrent job will continue running, causing other jobs to wait in queue until the parallel pool is shut down.

MATLAB: Calling parpool with SpmdEnabled = False

Unfortunately, only MJS and Local cluster types support SpmdEnabled = false. You might be able to use the "cluster parfor" approach though - see the documentation. Basically, you would transform your main parfor loop like so:

% Important: do *not* create a parallel pool prior to running this!
% In fact, you may wish to call "delete(gcp('nocreate'))" to be sure.
% Get the cluster object
c = parcluster('HPCServerProfile1');
% Create the parforOptions structure
opts = parforOptions(c);
% Run the parfor loop directly on the cluster with no parallel pool
parfor (idx = 1:10000, opts)
    % Body of the parfor loop
end

This should be more robust, however it does incur more overhead than the interactive parpool approach.

Best Answer

Related Solutions

MATLAB: Is the communicating job stuck behind a concurrent job in queue in R2017b

MATLAB: Calling parpool with SpmdEnabled = False

Related Question