MATLAB: How to take advantage of AWS Ec2 if the code cannot be parallelized

awsec2MATLABparallel computing

I launched an EC2 instance to run my Matlab code faster. Despite the fact that I chose a machine much more powerful than my Mac, the code ran slower on AWS.

The problem seems to be that my code is not parallelized. Apparently, while using my Mac Matlab will automatically parallelize certain operations, it does not on AWS (this is what I understood, does it make sense? Why should that be the case?).

I parallelized my code to find out that the parallelized version runs slower on my Mac. After some research I came to the conclusion that there might be two reasons for that:

When I parallelize (e.g., use a parfor) each worker operates with one core and this shuts down the "automatic parallelization" of Matlab
the part that takes the most time in my code is a loop in which each operation takes little time, but the loop is long (many of these iterations). As each operation does not take a lot of time, the overhead time outweighs the "benefits".

My question is: how can I take advantage of EC2 if parallelizing my code does not solve the problem?

Why is multi-threading off on EC2? Can I activate it somehow?

Best Answer

Edit the cluster profile to increase the number of threads per worker. The parpool size must be reduced by the same proportion.

You might also need to use coarser grained parallelism, move the parfor further back. This might require manually partitioning the problem into sections that can be calculated independently, followed by some "stich-up" work to handle boundary conditions. https://www.mathworks.com/help/parallel-computing/discover-clusters-and-use-cluster-profiles.html

Related Solutions

MATLAB: Use both multi core CPUs

You need to edit the 'local' parallel profile to set the maximum number of workers to 32. See the doc. Once you've done that, you should be able to issue the following command:

parpool('local', 32)

and launch 32 workers.

Note that if MATLAB by default does not consider hyper-threaded cores, as these tend not to perform well for numerically intensive computations. So, you might well find that running 32 workers is actually slower than running 16, depending on the precise nature of your computation.

MATLAB: Parallel Matrix Multiplication on a Distributed Computing System

There are two levels of parallelism present in MATLAB:

Implicit Multi-threaded parallelism for certain built-in MATLAB commands, such as Matrix-Matrix Multiplication or Matrix Factorization.
Explicit parallelism present in Parallel Computing Toolbox

Let's focus on the implicit multi-threaded parallelism first.

If you have a multi-core or multi-processor machine then the implicit multi-threaded parallelism is on by default in the client MATLAB. When I use the term client MATLAB, I mean the interactive MATLAB that you are running on your Windows/Linux/Mac desktop.

The number of threads used is set automatically by MATLAB at run time. You can type

>>maxNumCompThreads

to find out how many threads MATLAB is using for computation.

Keep in mind that MATLAB ignores hyperthreading. So for example if you have a hyperthreaded processor, your operating system might report 8 cores, but MATLAB will only see the 4 physical cores and report 4 as result of maxNumCompThreads.

If you need to you can disable implicit MATLAB multi-threading using one of the following:

Start Start MATLAB with -singleCompThread startup option
Type maxNumCompThreads(1) in your program

Note that maxNumCompThreads is currently deprecated and could be discontinued in a future release of MATLAB.

http://www.mathworks.com/help/releases/R2011b/techdoc/ref/maxnumcompthreads.html http://www.mathworks.com/help/releases/R2011b/techdoc/ref/matlabwindows.html

In relation to the earlier answer, if you start MATLAB on a machine that has 192 cores, MATLAB will report maxNumCompThreads of 192. On such a large machine the client MATLAB will have implicit parallelism of 192 threads. However, you as a user will not be able to control on which cores the threads are run. That will be handled by the operating system.

Now let's discuss the explicit parallelism provided by the Parallel Computing Toolbox and MDCS.

When you type MATLABPOOL open, MATLAB starts instances of headless MATLAB workers. These workers run either on your local machine (local scheduler) or on a MDCS cluster. These workers are by default single-threaded.

You can test that using the following code snippet:

matlabpool open
spmd
   maxNumCompThreads
end
matlabpool close

However, if you believe that your application would benefit from using a hybrid of explicit and implicit parallelism, for example your tasks are performing many matrix multiplications or matrix factorizations, you can re-enable the implicit parallelism by placing

maxNumCompThreads(N)

at the top of the function you are trying to run on workers, or inside spmd block.

spmd
   maxNumCompThreads(N);
end

In my example N is the number of threads that you want to use and should be a reasonable value, for example 2, 4, 8.

You should only re-enable implicit parallelism on workers in situations where it is really warranted, for example, if you have a single MATLAB worker on a multi-core node.

If you have multiple MATLAB workers running on a single node, I would not recommend enabling multi-threaded support as this will most likely result in performance degradation. See Cleve’s article for benchmarks related to mixing multi-threading and workers: http://www.mathworks.com/company/newsletters/news_notes/june07/clevescorner.html

At the moment it is still possible to enable multi-threading on the workers using the maxNumCompThreads command. However, this functionality is deprecated and could be removed in a future release of MATLAB.

Best Answer

Related Solutions

MATLAB: Use both multi core CPUs

MATLAB: Parallel Matrix Multiplication on a Distributed Computing System

Related Question