MATLAB: How to parallelize separate matrix-matrix multiplications

parallel computing

There are three matrix-matrix multiplications: C1=A1*B1, C2=A2*B2, C3=A3*B3.

If I wrote the code like above, the three matrix-matrix multiplications will be calculated in a sequential way. Is there an easy way to parallelize the three matrix-matrix multiplications (apart from using parfor)?

Apology if this question has been raised&answered before. I didn't find such a question after some quick search.

Best Answer

Unless you are working on a cluster, I would not expect to see any speed up from parallelizing such operations. The reason for this is that a matrix multiplication is multi-threaded, meaning that each of these computations is already utilizing multiple cores on your machine.

If you are working on a cluster, you can assign each of these variables in an SPMD block and perform the computation on each machine separately.

Related Solutions

MATLAB: Matrix operations with Parallel Computing Toolbox

Most math operations are inherently multithreaded. There is a list you will find on our external page:

http://www.mathworks.com/support/solutions/en/data/1-4PG4AN/

Jill will correct me if I am wrong, but each worker in the matlab pool is always single threaded. Which means the multithreaded operations will run single threaded on a worker but you have the advantage to scale to a large number of machines.

You may also find this link helpful which explains the difference.

http://www.mathworks.com/discovery/multicore-matlab.html

MATLAB: How can i speed up this large matrix multiplications

There is also another way to precompute B*C as explained my answer on this similar question

It's actually very memory consuming for full matrix and noticeable slower due to inefficient memory access, even the FPU requirement is less.

However in some cases of SPARSE matrices, it can win, as showed in the test case bellow:

m = 5000;
n = 1000;
p = 10000;
C = sprand(p,m,1e-2);
B = sprand(p,n,1e-2);
A = rand(p,1);
tic
(A.*B).'*C;
toc % Elapsed time is 0.186504 seconds.
% Preparation step, must be carried out only once, since B and C are known
p = size(B,1);
[ib,jb,b] = find(B.');
[ic,jc,c] = find(C.');
commonj = intersect(jb,jc);
keepb=ismember(jb,commonj);
keepc=ismember(jc,commonj);
ib = ib(keepb);
jb = jb(keepb);
b = b(keepb);
ic = ic(keepc);
jc = jc(keepc);
c = c(keepc);
[~,locb]=ismember(jb,commonj);
[~,locc]=ismember(jc,commonj);
cb = accumarray(jb,ib,[p 1],@(x) {x});
cc = accumarray(jc,ic,[p 1],@(x) {x});
ifun = @(cb,cc) reshape(cb(:)+(cc(:).'-1)*size(B,2),[],1);
ibc = cellfun(ifun, cb, cc, 'unif', 0);
jfun = @(j) j+zeros(size(ibc{j}));
jbc = arrayfun(jfun, (1:p).', 'unif', 0);
b = accumarray(jb,b,[p 1],@(x) {x});
c = accumarray(jc,c,[p 1],@(x) {x});
vfun = @(b,c) reshape(b(:).*(c(:).'),[],1);
vbc = cellfun(vfun, b, c, 'unif', 0);
ibc = cat(1,ibc{:});
jbc = cat(1,jbc{:});
vbc = cat(1,vbc{:});
BC = sparse(ibc,jbc,vbc,size(B,2)*size(C,2),p);
%%this should be in the loop 
tic
reshape(BC*A,size(B,2),[]);
toc % Elapsed time is 0.072390 seconds.

Best Answer

Related Solutions

MATLAB: Matrix operations with Parallel Computing Toolbox

MATLAB: How can i speed up this large matrix multiplications

Related Question