MATLAB: Parallel.pool.Constant: An error occurred building a parallel.pool.Constant.

parallel computingParallel Computing Toolboxparallel.pool.constant

I am trying to run a script which is of the type below:

            ni=670; 
            N=100;
            z = randn(512,512,100,ni*N);
            C = parallel.pool.Constant(z);
            parfor nn=1:N
                a{nn} = function1( C.Value(:,:,:,(nn-1)*ni + 1:nn*ni) + d{nn}, lambda1 );
                d{nn} = function2( d{nn} + C.Value(:,:,:,(nn-1)*ni + 1:nn*ni) );
                temp{nn} = function3( a{nn} - d{nn}, lambda2 );
            end

However I am getting the following error:

        *Error using parallel.pool.Constant (line 177)
        An error occurred building a parallel.pool.Constant.
        Caused by:
        No workers are available for FevalQueue execution.
        A write error occurred while sending to worker 6.*

Could anybody help with why this is happening and what can I do to resolve it?

System Config:

     CPU: Intel(R) Xeon(R)CPU E5-2687Wv4 @3.00GHz
     RAM: 512GB 
     OS: Windows 10 Pro 64-bit
     Checked on MATLAB R2016a and R2016b

thanks, Bis

Best Answer

That error means that a worker crashed for some reason, I strongly suspect out-of-memory. Note that your z matrix is huge - 1400GB; also note that parallel.pool.Constant still requires a copy of z on each worker (the benefit of parallel.pool.Constant is not that it reduces overall memory usage, rather that it stops you having to transfer data multiple times).

I think it should be possible to slice z, rather than use a parallel.pool.Constant. If you reshape z, you should be able to pick whole slices using an expression something like

    z(:,:,:,:,nn)

That way, each worker only receives the portion of z it requires. This will drastically reduce the memory usage required.

Related Solutions

MATLAB: Can the Parallel Computing Toolbox and the MATLAB Distributed Computing Server take advantage of nondistributed systems such as multi-CPU systems

For MATLAB R2016b onward, the Parallel Computing Toolbox (PCT) provides the ability to run up to 512 local MATLAB “workers” on the same machine to perform computations. These workers can potentially leverage the additional computing power available in multi-core and multi-processor machines. Note that these worker processes are associated with the session of MATLAB from which they were invoked as well as the user who invoked these processes. As a result, the worker processes can not be accessed by multiple users.

As of MATLAB R2016b, PCT workers can be run in multithreaded mode so you can increase the number of computational threads without increasing the number of workers. This is seen in the following MATLAB release notes:

http://www.mathworks.com/help/distcomp/release-notes.html#bvczl_g-1

MATLAB itself has no limit on the number of computational threads. However, it is important to note that it is recommended to not have more computational threads from all MATLAB processes on a system than the number of physical cores.

Therefore, let's say you have 2 CPUs, 8 cores/CPU and so there are 16 physical cores. This setup can be used by any one of the following:

• A single MATLAB client with 16 computational threads

• 16 PCT local workers, each running single-threaded

• 8 PCT local workers, each running with 2 computational threads

• 4 PCT local workers, each running with 4 computational threads

For more information, refer to the following MATLAB documentation: <http://www.mathworks.com/help/distcomp/discover-clusters-and-use-cluster-profiles.html#f5-16540>

For MATLAB R2016a and earlier, local MATLAB workers can only run in single-threaded mode.

For MATLAB R2013b and earlier, the number of local workers is limited to twelve.

Alternatively, the MATLAB Distributed Computing Server can be used to start up a job manager and multiple workers on a multi-core or multi-CPU system. Even in the case of computing clusters that contain a mix of single CPU and multiple CPU machines, workers can be utilized on a per-core or per-CPU basis. Refer to the Related Solution for some guidelines to help you determine how many worker sessions to run on one computer.

When executing parallel jobs, workers communicate using MPI even if they are running on the same physical computer. However, you can choose to use a version of MPI that is optimized for Shared-Memory Processor systems.

Refer to the documentation for the mpiLibConf function (enter the following at the MATLAB command prompt)

 web([docroot, '/toolbox/distcomp/mpilibconf.html'])

for information on using alternative versions of MPI instead of the build of MPICH2 that the MATLAB Distributed Computing Server uses by default.

MATLAB: Using parallel.pool.constant with parfeval

You need to explicitly pass the instance of parallel.pool.Constant into your parfeval call - it doesn't work like a global variable - it works more like an ordinary variable, but with some behind-the-scenes intelligence to minimise data transfers. So, here's what you'd need to change:

% In your script:
...
    F(ab) = parfeval(@Test2, 1, Testp);
...
% In Test2.m:
function [out] = Test2(inConst)
    out = inConst.Value; 
end

Best Answer

Related Solutions

MATLAB: Can the Parallel Computing Toolbox and the MATLAB Distributed Computing Server take advantage of nondistributed systems such as multi-CPU systems

MATLAB: Using parallel.pool.constant with parfeval

Related Question