MATLAB: Passing big matrix to workers

parallel computingParallel Computing Toolboxparfor

I'm trying to pass very large matrix (65000x460x150) to workers with Worker Object Wrapper, with no luck.

The code:

Imat_worker = WorkerObjWrapper(Imat);
parfor kk = 1:size(num_iterations,2)
   Imat_par = Imat_worker(:,:,1:num_iterations(kk));
   some code...
end

I get the following message while running the code:

Error using WorkerObjWrapper/workerInit (line 156)
The parallel pool that SPMD was using has been shut down.
Error in WorkerObjWrapper (line 97)
            WorkerObjWrapper.workerInit( tmpId, ctor, args, dtor ); 
A write error occurred while sending to worker 2.

Any ideas what am I doing wrong?

It is worth noting that I'm running the code on a server with enough RAM for the matrix.

Thanks in advance!

Best Answer

In that case, since I need only a slice of the matrix for each worker, shouldn't I clone only the relevant data somehow before the parfor ?

Yes you should, and that's something parfor will do for you automatically as long as the operations you do on Imat inside the loop satisfy certain restrictions.

Related Solutions

MATLAB: Parallel.pool.Constant: An error occurred building a parallel.pool.Constant.

That error means that a worker crashed for some reason, I strongly suspect out-of-memory. Note that your z matrix is huge - 1400GB; also note that parallel.pool.Constant still requires a copy of z on each worker (the benefit of parallel.pool.Constant is not that it reduces overall memory usage, rather that it stops you having to transfer data multiple times).

I think it should be possible to slice z, rather than use a parallel.pool.Constant. If you reshape z, you should be able to pick whole slices using an expression something like

    z(:,:,:,:,nn)

That way, each worker only receives the portion of z it requires. This will drastically reduce the memory usage required.

MATLAB: ‘parfor’ runs sequentially when using large dataset

Unfortunately, the process of copying the data to the workers makes several temporary copies, so your 64GB memory is not enough for copying 4*10GB data to parallel workers. When parallel pool don't have enough memory to initialize, it will turns to run the 'parfor' loop sequentially, which is the behavior you are observing.

As possible workarounds:

1) If thread workers are sufficient, then this might possibly avoid the data duplication. I.e. use

parpool('threads')

\nbefore running the 'parfor' loop.\n \n2) With process workers (i.e. "parfor('local')"), then 'parallel.pool.Constant' might help. Here's how you would do that:

data = parallel.pool.Constant(@createLargeArray);
parfor i = 1:3000
   out(i) = doStuff(data.Value, i);
end

\nHere, I've assumed a function 'createLargeArray' which creates the 10GB data, and a function 'doStuff' that uses that data.

Best Answer

Related Solutions

MATLAB: Parallel.pool.Constant: An error occurred building a parallel.pool.Constant.

MATLAB: ‘parfor’ runs sequentially when using large dataset

Related Question