MATLAB: How to use multiple GPUs to solve one algebraic system

Question

I have multiple GPUs. How to use these GPUs to solve one algebraic system? Thx!

Answer 1

Mostly you cannot.

Howeve in some cases you can manually segment out the work to be done into multiple processes using parfor() or parfeval() . Each worker will automatically be allocated a different GPU (provided there are as many GPUs as workers.)

You cannot access multiple GPUs from the same worker -- or rather every time you do, using gpu device selection, you reset the GPU that was being used and it stops the work it was doing.

When I say manually segment out the work, I mean that in some cases it can make sense to work on a smaller part of the problem and combine the results later. It does not work to just do something like

N = 1e4;   %must be even for this example

A = gpuarray(rand(N, N));
parpool 2
b = eig(A);

This will not distribute the eig work over two GPUs!

But sometimes for some operations you can do things like

N = 1e4;   %must be even for this example
A = rand(N, N);
Ap{1} = A(1:N/2, 1:N/2);
Ap{2} = A(1:N/2, N/2+1:N);
Ap{3} = A(N/2+1:N, 1:N/2);
Ap{4} = A(N/2+1:N, N/2+1:N);
Ep = cell(1,4);
parpool 2
parfor K = 1 : numel(Ap)
    ga = gpuArray(Ap{K});
    ge = SomeFunction(ga);
    Ep{K} = gather(ge);
end

and then combine the results that are in Ep