MATLAB: Am I unable to use parpool with the local scheduler or validate the local configuration of Parallel Computing Toolbox

MATLABParallel Computing Toolbox

When attempting to use parpool in Parallel Computing Toolbox, it will not open or I receive an error message. I am unable to validate my local profile.

Best Answer

There are several issues that can prevent the parpool from starting. Run the following tests below to make sure that your configuration is setup properly. If at any point you receive an error message, you can submit a request to Installation support using the link at the bottom of the page. When submitting a request, be sure to include the following:

- Your license number

- The release of MATLAB and PCT

- The output of your validation (click details to get the full information)

- The results of the tests below

Also when submitting a request please reference Solution 1-C27YO8.

NOTE: If you are using R2008a and you are unable to use Parallel Computing Toolbox, see the bug report here:

http://www.mathworks.com/support/bugreports/details.html?rp=454732

1) Make sure your license of Parallel Computing Toolbox works

In MATLAB you can run the following command to check your license:

license checkout Distrib_Computing_Toolbox

You will receive the output "ANS=1" if the license is working. Otherwise, you will see a license manager error. In that case, searching for the license error message should give a solution for how to solve your issue.

2) Make sure your release of MATLAB matches the release of PCT

To check the release of your products, run the "ver" command in MATLAB. Next to each product there should be a release (ex: R2009b). Make sure that each release matches (It is fine if there is plus "+" next to some of the products and not all)

If the release of Parallel Computing Toolbox and MATLAB do not match the release of MATLAB and your other products, you will not be able to use this configuration until the installations are at the same release.

3) Disable local mpiexec

If you are using R2010a or newer, you may experience issues with the new local mpiexec implementation. In that case, try the following command to disable this feature:

distcomp.feature( 'LocalUseMpiexec', false )

This should allow the local scheduler to create and process parallel jobs and parpool.

4) Check your local scheduler configuration

There are no changes that need to be made in order to use the local scheduler, but if you have made changes to the configuration, you may want to reset these. This can be done by creating a new local scheduler configuration. To do so,

1. Go to the Parallel menu in MATLAB and select "Manage Cluster Profiles..." ("Manage Configurations..." for R2011b or earlier)

2. Click on Add > Custom > Local (for older releases: From the File menu, select New > Local Configuration)

3. Click the radio option in the default column to set this as the default configuration

Once complete, close the Manage Configuration windows and try again.

5) Clear the local scheduler data folder

The error you are seeing might be the result of bad local scheduler data. In that case, the local scheduler data can be removed. To do so:

1. In MATLAB, run the command "prefdir" to find your preferences folder. Ex:

>>prefdir
ans =
C:\Users\Administrator\AppData\Roaming\MathWorks\MATLAB\R2009b

This will output your preferences folder. The local scheduler data folder is one level up from the preferences in a folder called "local_scheduler_data" or "local_cluster_jobs". For example:

C:\Users\Administrator\AppData\Roaming\MathWorks\MATLAB\local_scheduler_data

2. Close MATLAB

3. Rename or delete the local_scheduler_data or local_cluster_jobs

4. Restart MATLAB

You can now try to run parpool to see if the data folder was the problem.

6) Ensure that hostname resolution works on your computer

In order to use the local scheduler, your computer's own hostname must be resolvable. To confirm this, run the following command in MATLAB:

!hostname

This will give you your computer's hostname. You must be able to resolve this hostname to the computer's IP address. To test this you can run:

!ping <hostname>

Where <hostname> is the output of the hostname command above. If the results indicate the wrong IP address or say that your computer is an "unknown host", there is a network issue on your computer that needs to be resolved in order to use the local scheduler. In that case, ask your network administrator for help.

7) Check to see if you have either:

- a startup.m on the MATLAB path or

- one or more additional pathdef.m or a matlabrc.m scripts shadowing the correct versions that are stored in <matlabroot>/toolbox/local

Any of these things may cause an error or just cause parpool and validation to just hang, even if it works fine in MATLAB when run as code. Sometimes these problems only appear after a customer has upgraded to a new version of MATLAB. .

To determine if any of these files are a problem, run the following commands in MATLAB Command window.

>> which -all startup.m
>> which -all pathdef.m
>> which -all matlabrc.m

If any pathdef or matlabrc files are present that are shadowing the <matlabroot>/toolbox/local version, then either remove them from the path or move their file path down the MATLAB Path list so that it sits beneath the <matlabroot>/toolbox/local path.

If any startup file is present, remove it from the MATLAB path.

If you are still unable to run parpool, run a validation of your local scheduler configuration and submit this to support. To validate:

1. Go to the Parallel menu in MATLAB and select "Manage Cluster Profiles..." ("Manage Configurations..." for R2011b or earlier)

2. Highlight your local scheduler configuration and click the "Validate" button ("Start Validation" for R201b or earlier)

3. Once the validation completes, click the "details" link to see the results

You can then forward your output of validation, the results of the tests below, and your license number to support here:

https://www.mathworks.com/support/contact_us.html

Related Solutions

MATLAB: Am I unable to validate the LSF configuration in the Parallel Computing Toolbox

There are several issues that can prevent the validation of the cluster. Run the following tests below to make sure that your configuration is setup properly. If at any point you receive an error message, you can submit a request to Installation support using the link at the bottom of the page. When submitting a request, be sure to include the following:

- Your license number

- The release of MATLAB on the client and the cluster

- The output of your validation (click details to get the full information)

- The results of the tests below

Also when submitting a request please reference Solution 1-BJSCEG.

1) Test the licensing of MATLAB Parallel Server

The first step is to ensure that the licensing for MATLAB Parallel Server works on your cluster. This will also test to see if MATLAB is crashing on startup on your cluster. To test this, go to one of the cluster nodes and open up a Windows Command Prompt by clicking on the Start Menu and go to All Programs, Accessories, and click on Command Prompt. In the command prompt, run the following commands:

cd $MATLAB\bin (where $MATLAB is the installation folder for MATLAB on the cluster)

./matlab -dmlworker -nodisplay -logfile /var/tmp/output.txt -r "ver;exit"

This will generate an output.txt file in /var/tmp that contains the ver output on the cluster. If the log file contains a network license manager error, this is the issue. In that case, check the support site for the license manager error number and take the appropriate action to resolve the license error before proceeding.

2) Check the releases of MATLAB on the cluster and the client where you validated

If you get the output of the "ver" command in the log file, check the releases of all the products in the list. The release of each product should match for all the products. Additionally, the release should match the release that is installed on the client where you ran the validation. To check the release on the client, run the ver command in MATLAB's command window. If the release of Parallel Computing Toolbox and MATLAB do not match the release of MATLAB and MATLAB Parallel Server on the cluster, you will not be able to use this configuration until the installations are at the same release.

3) Check to make sure that your configuration meets the scheduler requirements

In order to use MATLAB Parallel Server with LSF, there are some additional requirements in the setup. Check the scheduler requirements page here for more details:

https://www.mathworks.com/support/requirements/matlab-parallel-server.html

Additionally, this configuration requires the following:

- The LSF binaries need to be accessible from the MATLAB client that runs Parallel Computing Toolbox. If the client does not have the binaries, it is recommended to remotely access one of the cluster nodes to run the MATLAB client.

- Your cluster should be completely homogeneous. Mixing different platforms or distributions is not recommended especially for parallel computation.

- This configuration requires that the data for the jobs be stored on a shared file space between the clients and the cluster nodes. When creating the configuration, set the "DataLocation" variable to be a path that is accessible to all computers.

- Since the "DataLocation" variable needs to be accessible by the same path from all computers, you cannot use a client machine of a different platform (Ex: Running a Windows client to access a Linux cluster).

If the 4 requirements above are not met, the default LSF configurations are not supported. In that case, it is still possible to submit jobs to the cluster. For this setup, see the related solution: 1-34TP79 - "How can I use the MATLAB Distributing Computing products with the LSF scheduler in a nonshared file system?"

4) Check to ensure you have correctly configured the client configuration

In your client MATLAB, go to the Parallel menu to Manage Configurations. Right click on your LSF configuration and select Properties. You must set the appropriate values for ClusterMatlabRoot (the directory where is MATLAB installed on the cluster), DataLocation (where the data will be stored, NOTE: This must be accessible from the same path from all computers), ClusterOsType (unix or pc), and HasSharedFilesystem (should be set to True).

If you have a cluster that has a mixture of different operating systems, you must use the "SubmitArguments" field to target only one type of operating system. For example:

‘-R “type==NTX86”’ - Targets 32-bit Windows

‘-R “type==NTX64”’ - Targets 64-bit Windows

‘-R “type==LINUX86”’ - Targets 32-bit Linux

‘-R “type==LINUX64”’ - Targets 64-bit Linux

NOTE: Run the "lshosts" command on the cluster and use the "type" column to find the type that you should target.

If you have confirmed all of the settings above, do all stages fail during validation, or just parallel and Matlabpool? If you are able to pass the Distributed Job phase, the validation may be reporting false errors. To confirm you can manually validate your cluster. To do so:

1. Distributed job:

To run a simple distributed job, run the following:

lsf = findResource('scheduler','configuration','<ConfigurationName>')

Where <ConfigurationName> is the name of the configuration you created

job = createJob(lsf);
createTask(job, @sum, 1, {[1 1]});
createTask(job, @sum, 1, {[2 2]});
createTask(job, @sum, 1, {[3 3]});
submit(job)
waitForState(job, 'finished', 60)

To confirm the job completed, run the following:

results = getAllOutputArguments(job)

If you get the following output, your cluster is configured and operating correctly.

results =
[2]
[4]
[6]

2. Parallel job:

After completing the distributed job, run the following:

pj = createParallelJob(lsf);
createTask(pj, @labindex, 1, {});
set(pj, 'MaximumNumberOfWorkers', 3);
set(pj, 'MinimumNumberOfWorkers', 3);
submit(pj)
waitForState(pj, 'finished', 60)

To confirm the job completed, run the following:

results = getAllOutputArguments(pj)

If you get the following output, your cluster is configured and operating correctly.

results =
[1]
[2]
[3]

3. MATLAB pool job:

To test MATLABPool or PMODE, simply run the command:

matlabpool open <ConfigName> <#ofLabs>

Where <Configname> is the name of the configuration and <#ofLabs> is the number of nodes to use in the cluster.

If your prompt is returned, your configuration is working. To quit MATLAB pool, simply type "exit".

If the MATLAB pool did not start and you did not receive an error message, try running:

setSchedulerMessageHandler(@disp)

and then try the MATLAB pool commands above. This should capture the error messages and forward them to the MATLAB command window.

If the manual tests passed, your configuration is working and you should be able to submit jobs.

If you are still having an issue, contact Installation support here:

https://www.mathworks.com/support/contact_us.html

NOTE

: Starting in R2019a the following name changes occurred:

MATLAB Distributed Computing Server was renamed to MATLAB Parallel Server
mdce_def was renamed to mjs_def
mdce binary was renamed to mjs

MATLAB: Error opening matlabpool

Could you try to add localhost with 127.0.0.1 to your /etc/hosts file?

Titus

Best Answer

Related Solutions

MATLAB: Am I unable to validate the LSF configuration in the Parallel Computing Toolbox

MATLAB: Error opening matlabpool

Related Question