MATLAB: Using rtwbuild in parallel is not recommended

Parallel Computing Toolboxrtwbuildsimulink

In the documentation for rtwbuild, there is a note: 'Do not use rtwbuild, rtwrebuild, or slbuild commands with parallel language features'. What is the reason for that? I wanted to speedup code generation for several models by using parallel toolbox, and it works. However the rtwbuild is slower on the workers than on the client for the same models, thus the speedup is lower than I expected. Could anyone explain/suggest how rtwbuild works on the client and on workers?

Best Answer

Model build generates several artifacts (e.g., several .mat files) for a model which are used in the code generation process. So if you are using one of these commands to build a single model multiple times via multiple threads, then these threads will run into resource contention for these artifacts. But from your explanation I assume you are building multiple models which are independent of each other. In this case, you can use rtwbuild with parallel computing features.

Now, there could be multiple factors affecting the speed of the build in workers. Are you running the workers locally or in a remote machine? - If they are running locally in the machine same as the client, then the increased overload on the machine's memory and CPU load could be slowing down the build. - If they are running remotely, then the remote machine's ability (memory and CPU load) to handle memory (or any resource) contention by parallel processes efficiently matters.

Having said all these, the one important information that you should consider is "How slower are these build?". So before diagnosing the effect of all these factors, please note that, in programming world, parallel computation is not always guaranteed to be faster than sequential computations. There would be scenarios where the parallel computations could be a bit slower than sequential.

In general, rtwbuild is designed for single-threaded application and hence there is no difference in the way it works on client and on workers.

Lastly, if you are using model references in your model, we do support parallel build of these references. For more information on this : https://www.mathworks.com/help/rtw/ug/reduce-build-time-for-referenced-models.html

Related Solutions

MATLAB: Referenced model Makefiles are not generated for rsim parallel build with GenCodeOnly disabled

The Makefiles generated and executed on the parallel workers are invalid when they are copied back to the client, as all the paths in the Makefiles are relative to the temporary build folders used by the workers. In previous releases, the Makefiles were regenerated on the client during the merge phase, but this is expensive and unnecessary for the parallel build to succeed.

When "GenCodeOnly" is on, where the intention is to compile the code at a later point, the Makefiles are still regenerated. However, when "GenCodeOnly" is off, the Makefiles are discarded which gives a performance boost depending on the size/structure of the model reference hierarchy.

One possible workaround is to regenerate the Makefiles with the "rtwrebuild" command:

https://www.mathworks.com/help/rtw/ref/rtwrebuild.html

Another workaround is to call "rtwbuild" twice, once with "GenCodeOnly" on, and once with "GenCodeOnly" off:

rtwbuild('model_name'); % GenCodeOnly='on';
rtwbuild('model_name', 'generateCodeOnly', false);

MATLAB: Use both multi core CPUs

You need to edit the 'local' parallel profile to set the maximum number of workers to 32. See the doc. Once you've done that, you should be able to issue the following command:

parpool('local', 32)

and launch 32 workers.

Note that if MATLAB by default does not consider hyper-threaded cores, as these tend not to perform well for numerically intensive computations. So, you might well find that running 32 workers is actually slower than running 16, depending on the precise nature of your computation.

Best Answer

Related Solutions

MATLAB: Referenced model Makefiles are not generated for rsim parallel build with GenCodeOnly disabled

MATLAB: Use both multi core CPUs

Related Question