MATLAB: Can I use GPU computing inside of a parfor loop

Question

I am in the process of upgrading my computer and I am wondering if there is a potential for upgrading my GPU. Right now I am only using parfor. However, within the parfor loop there are operations that would seem to benefit from using GPU computing. I get the impression from this entry ( https://www.mathworks.com/matlabcentral/answers/36235-parfor-on-gpu ) that for what I want to do, I would need to use multiple GPUs. Now, I don't know enough about GPUs to know what kind of questions to ask, but I'll start with these:

If I am running 8 workers with parfor, does this mean that I would need to allocate 8 GPUs? Does GPU = graphics card and if so, then I need to buy 8 graphics cards (impractical)? Or do graphics cards have multiple GPUs? If this is possible, how would I go about choosing the right graphics card for the job?

Thanks in advance -jz

Answer 1

You do not need 8 separate graphics cards even if you have 8 workers using gpuArrays simultaneously. (In fact, I'm not even sure that you can put 8 GPUs in a single system). Having multiple workers using a single GPU simultaneously is called "oversubscribing". Whether or not you see any performance benefit from this depends strongly on the details of your problem - and how much of that you can actually run on the GPU.

For example, consider a case where you have a problem that, when run in a single MATLAB session, only spends 1/4 of its time using the GPU. In that case, you might expect good performance running 4 workers accessing the same GPU (providing the arrays all still fit in memory). If you run that against 8 workers, you will see diminishing returns as the workers all contend for access to the GPU.