MATLAB: Can I submit to a MATLAB Parallel Server cluster that is at a different release number than the MATLAB and PCT setup

MATLABMATLAB Parallel ServerParallel Computing Toolbox

I have a MATLAB Parallel Server cluster that is either newer or older than the version of MATLAB I have installed on my client machine. Can I still utilize this client machine to submit jobs to my MATLAB Parallel Server setup?

Best Answer

For R2015b and earlier, in order to submit a job to the cluster, the versions of MATLAB on the client machine and MATLAB Parallel Server on the cluster must match.

From MATLAB R2016a onwards, you can upgrade your MATLAB Job Scheduler-based clusters to a more recent Release of MATLAB Parallel Server but continue to use older versions of Parallel Computing Toolbox to submit jobs to it . To take advantage of this backward compatibility feature:

Install the latest version of MATLAB Parallel Server on your cluster. You must use this version to run MATLAB Job Scheduler on your cluster.
Install MATLAB Parallel Server for each release that you want to support with the cluster. For example, to use Parallel Computing Toolbox R2016a and R2016b with your cluster, install both the R2016a and R2016b releases of MATLAB Parallel Server.
Configure MATLAB Job Scheduler with the location of these installations. In the mjs_def configuration file, specify the location of each installation of MATLAB Parallel Server in the MJS_ADDITIONAL_MATLABROOTS variable. You can find this file in matlabroot/toolbox/parallel/bin for Linux (mjs_def.sh) and Windows (mjs_def.bat). For more information, see mjs.

With this configuration, the MATLAB Job Scheduler allows MATLAB clients from the installed releases to submit jobs to the cluster. The MATLAB Job Scheduler dynamically starts the right version of the MATLAB worker to run the job.

NOTE: Starting in R2019a the following name changes occurred:

MATLAB Distributed Computing Server was renamed to MATLAB Parallel Server
mdce_def was renamed to mjs_def
mdce binary was renamed to mjs
mjs_def.bat file is in $MATLAB/R20XXx/toolbox/distcomp/bin for R2019a and earlier

Related Solutions

MATLAB: How to start the MDCE in MATLAB 7.5 (R2007b) or later versions

Below is a sample M-script for starting up 4 workers on a local machine with the MATLAB Distributed Computing Engine installed.

%%This document assumes you have successfully installed the MDCE Toolbox
%%Install the MDCE
cd([matlabroot '\toolbox\distcomp\bin'] or (matlabroot '\toolbox\parallel\bin')(R2019b & later)
!mdce install
%%Start the MDCE
!mdce start
%%Start the MDCE and prepare for Distributed processing
% If you would like to see the status of the workers as they run, please
% set this to true
STATUS = true;
% Start a Job Manager
% -name the name of the Job Manager
!startjobmanager -name my_jobmanager
% Start 4 workers using the previously defined Job Manager
% -name           name of the worker
% -jobmanager     name of the Job Manager
% -jobmanagerhost name of the machine the manager is running on
!startworker -name worker_1 -jobmanager my_jobmanager -jobmanagerhost localhost
!startworker -name worker_2 -jobmanager my_jobmanager -jobmanagerhost localhost
!startworker -name worker_3 -jobmanager my_jobmanager -jobmanagerhost localhost
!startworker -name worker_4 -jobmanager my_jobmanager -jobmanagerhost localhost
if STATUS
% Output the status of the workers as they are running
!nodestatus
end
%%Run your code
% After the above has been started, you can run your code
% When you are done running your code execute the below cells of code
% to cleanly shut down the Workers, Job Manager and MDCE
%%Stop all the Workers, Job Manager and the MDCE
cd([matlabroot '\toolbox\distcomp\bin'])
% Stop the 4 workers previously started
!stopworker -name worker_1 -jobmanager my_jobmanager
!stopworker -name worker_2 -jobmanager my_jobmanager
!stopworker -name worker_3 -jobmanager my_jobmanager
!stopworker -name worker_4 -jobmanager my_jobmanager
% Stop the Job Manager
!stopjobmanager -name my_jobmanager
%%Stop the MDCE
!mdce stop
%%Uninstall the MDCE
!mdce uninstall

MATLAB: Do I receive an error when submitting jobs to HPC Server from a 32-bit Windows client

32-bit clients are supported for HPC starting with R2011a. For previous releases, see the winformation below.

Parallel Computing Toolbox only supports job submission to an HPC Server 2008 cluster from 64-bit Windows clients. In order to submit to this type of cluster, it must be done from a 64-bit Windows client. It is possible to submit from a 32-bit client, but this must be done by setting the "ClusterVersion" of the configuration to ccs. This allows the jobs to submit using the old CCS mode, which does not use the advanced HPC functions.

NOTE: Starting in R2019a the following name changes occurred:
MATLAB Distributed Computing Server was renamed to MATLAB Parallel Server
mdce_def was renamed to mjs_def
mdce binary was renamed to mjs

Best Answer

Related Solutions

MATLAB: How to start the MDCE in MATLAB 7.5 (R2007b) or later versions

MATLAB: Do I receive an error when submitting jobs to HPC Server from a 32-bit Windows client

Related Question