MATLAB: How to submit jobs to a specific LSF job queue on the Linux LSF cluster using Distributed Computing Toolbox 3.0

clustercomputingdistributedjoblsfParallel Computing Toolboxqueuespecificsubmit

I want to submit jobs to a specific LSF job queue on my Linux LSF cluster using Distributed Computing Toolbox 3.0.

Best Answer

This is possible to do using the 'SubmitArguments' property. For more information about this, please refer to the following documention online:

SubmitArguments :: Properties (Distributed Computing Toolbox)

https://www.mathworks.com/help/releases/R2012a/toolbox/distcomp/submitarguments.html

DCT invokes LSF via BSUB (the LSF scheduler command). As long as there is a command line argument that can specify what specific queue to submit to, you can pass this information along to BSUB via SubmitArguments.

Related Solutions

MATLAB: Can a worker registered with the MathWorks Job Manager be accessed from another scheduler (such as PBS)

A worker is a MATLAB session executing on a machine (node), and is associated with a job manager or with the scheduler that created it. Multiple schedulers can execute tasks on the same machine. For each scheduler accessing the machine, a different MATLAB worker session is started.

Each worker is dedicated to its job manager or scheduler, even if the same machine is being used by different schedulers. An idle worker that is associated with a job manager remains available (idle) to that job manager when another scheduler starts a worker on the same machine.

Note that a worker started by a third-party scheduler runs for only as long as it takes it to evaluate its task, then it is stopped. A worker that is registered with a job manager remains running and available between tasks.

MATLAB: How to launch a script as an LSF job from the Matlab console

Yes, you can use the batch command to do this. You'll need Parallel Computing Toolbox installed on your desktop MATLAB client, and MATLAB Distributed Computing Server installed on the cluster.

Instructions for configuring MATLAB Distributed Computing Server for an LSF cluster are here: https://uk.mathworks.com/help/mdce/configure-parallel-computing-products-for-supported-third-party-schedulers-pbs-pro-platform-lsf-torque.html

To set up specific options for your batch submission, you should set up SubmitArguments for your cluster object. You can do this either through the "Cluster Profile Manager", or at the command-line. Here's an example, presuming you've already set up your basic profile.

myCluster = parcluster('myLSFProfile');
myCluster.SubmitArguments = '-R rusage[mem=100]';
myJob = batch(myCluster, 'myScriptName');

This will submit myJob to your LSF cluster, to run the script called myScriptName, and it will add the specified SubmitArguments to the bsub command-line.

Best Answer

Related Solutions

MATLAB: Can a worker registered with the MathWorks Job Manager be accessed from another scheduler (such as PBS)

MATLAB: How to launch a script as an LSF job from the Matlab console

Related Question