MATLAB: Where are the details of the DataLocation property of a jobmanager object in the Parallel Computing Toolbox 4.2 (R2009b)

Parallel Computing Toolbox

I have noticed that when I execute the following code, where sch is a jobmanager object:

get(sch,’DataLocation’)

I receive the following output:

 database on scheduler_name@host

I would like to know where is the default location of the data files stored by the job.

Best Answer

The 'DataLocation' property of a scheduler object in the Parallel Computing Toolbox can point to the MDCE database. The database uses a location specified in the mdce_def.bat file to store the files it needs. By default this is usually C:\TEMP\MDCE on a windows machine, but it is sys-admin configurable (by changing the mdce_def.bat file).

All data (input and output) to jobs and tasks is stored in the database (including the FileDependencies for a job). Hence, with a lot of jobs and tasks, or if the job contains a lot of data, these files can become very large.

Related Solutions

MATLAB: Do I need to have a pre-existing “cluster” to use the Distrubuted Computing Toolbox and MATLAB Distributed Computing Engine

This enhancement has been incorporated in Release 2006a (R2006a). For previous product releases, read below for any possible workarounds:

The computers used for distributed computing do not require any pre-existing "clustering" software. The MATLAB Distributed Computing Engine (MDCE) enables computers on the same network to act as a cluster, in that tasks are distributed across several independent machines running the MDCE. An actual "cluster," running cluster management software, is not required to use the product.

MATLAB: How to configure a MATLAB worker in a cluster to have the security setting of a user who submits the job

By default, on Windows the workers run as a local system user, and on Linux and Mac OS X as root users. These users might have limited access to certain directories and network mounted drives. To change this setting you will need to modify the MDCE_USER property in mdce_def.bat file. You will need to set this property to a user who has read and write permissions to the group folder.

You will need to restart the MDCE daemon for the changes to take effect by executing the following commands:

 mdce stop
 mdce uninstall
 mdce install
 mdce start

Additionally, starting with MATLAB R2010b MATLAB Distributed Computing Server allows for a finer level of job security control, where each parallel job can be run as the user who submitted it to the cluster. To enable this, set the SECURITY_LEVEL to 3 in the mdce_def file. For more information on setting the job security level in MATLAB Distributed Computing Server please refer to the following documentation:

https://www.mathworks.com/help/releases/R2020a/matlab-parallel-server/set-matlab-job-scheduler-cluster-security.html

Best Answer

Related Solutions

MATLAB: Do I need to have a pre-existing “cluster” to use the Distrubuted Computing Toolbox and MATLAB Distributed Computing Engine

MATLAB: How to configure a MATLAB worker in a cluster to have the security setting of a user who submits the job

Related Question