MATLAB: How to determine number of workers

MATLABparallel computing

Let's say I have a parfor loop that contains N iterations (parfor ind = 1:N), which contains N independent calculations that I need. In this case, how can I determine the number of workers I shall call in the batch command [batch(@job, 'pool', M)]?

(1) Is it desirable to always have M = N? (2) What happens if M < N? Am I correct that in this case the remote cluster will run the first M jobs in parallel, and then go to the next M jobs, etc?

Thanks!

Best Answer

The number of workers in your batch parallel pool should not exceed the number of loop iterations, i.e. it is not useful to make M > N. However, if you have more loop iterations than workers, each worker will simply process multiple iterations. (This is exactly the same behaviour as when you create an interactive parallel pool using parpool(M)).

Related Solutions

MATLAB: How to take advantage of useParallel option for a remote job running on workers in a MDCS cluster

1. In order to use parallel pool with the cluster, please consider using the 'batch' function procedure to submit the jobs into cluster:

https://www.mathworks.com/help/distcomp/batch.html

When using the 'batch' command, it has the option to specify the number of workers to be used in addition to the job itself. Please refer to the example below:

c=parcluster('profile');
j = batch(c,@scriptUseParallel, 'Pool', 2);
wait(j);
diary(j);
Y2=fetchOutputs(j);

2. In the cluster setup, please consider to configure each machine to have the same number of workers as the number of cores.

Therefore each core can be started with a MATLAB process and being utilized by the batch job.

MATLAB: Can the Parallel Computing Toolbox and the MATLAB Distributed Computing Server take advantage of nondistributed systems such as multi-CPU systems

For MATLAB R2016b onward, the Parallel Computing Toolbox (PCT) provides the ability to run up to 512 local MATLAB “workers” on the same machine to perform computations. These workers can potentially leverage the additional computing power available in multi-core and multi-processor machines. Note that these worker processes are associated with the session of MATLAB from which they were invoked as well as the user who invoked these processes. As a result, the worker processes can not be accessed by multiple users.

As of MATLAB R2016b, PCT workers can be run in multithreaded mode so you can increase the number of computational threads without increasing the number of workers. This is seen in the following MATLAB release notes:

http://www.mathworks.com/help/distcomp/release-notes.html#bvczl_g-1

MATLAB itself has no limit on the number of computational threads. However, it is important to note that it is recommended to not have more computational threads from all MATLAB processes on a system than the number of physical cores.

Therefore, let's say you have 2 CPUs, 8 cores/CPU and so there are 16 physical cores. This setup can be used by any one of the following:

• A single MATLAB client with 16 computational threads

• 16 PCT local workers, each running single-threaded

• 8 PCT local workers, each running with 2 computational threads

• 4 PCT local workers, each running with 4 computational threads

For more information, refer to the following MATLAB documentation: <http://www.mathworks.com/help/distcomp/discover-clusters-and-use-cluster-profiles.html#f5-16540>

For MATLAB R2016a and earlier, local MATLAB workers can only run in single-threaded mode.

For MATLAB R2013b and earlier, the number of local workers is limited to twelve.

Alternatively, the MATLAB Distributed Computing Server can be used to start up a job manager and multiple workers on a multi-core or multi-CPU system. Even in the case of computing clusters that contain a mix of single CPU and multiple CPU machines, workers can be utilized on a per-core or per-CPU basis. Refer to the Related Solution for some guidelines to help you determine how many worker sessions to run on one computer.

When executing parallel jobs, workers communicate using MPI even if they are running on the same physical computer. However, you can choose to use a version of MPI that is optimized for Shared-Memory Processor systems.

Refer to the documentation for the mpiLibConf function (enter the following at the MATLAB command prompt)

 web([docroot, '/toolbox/distcomp/mpilibconf.html'])

for information on using alternative versions of MPI instead of the build of MPICH2 that the MATLAB Distributed Computing Server uses by default.

Best Answer

Related Solutions

MATLAB: How to take advantage of useParallel option for a remote job running on workers in a MDCS cluster

MATLAB: Can the Parallel Computing Toolbox and the MATLAB Distributed Computing Server take advantage of nondistributed systems such as multi-CPU systems

Related Question