MATLAB: How to run a PARFOR loop if the Parallel Computing Toolbox 4.1 (R2009a) does not have direct visibility of the cluster nodes

creatematlabpooljobMATLAB Parallel ServerParallel Computing Toolbox

I am using the Parallel Computing Toolbox 4.1 (R2009a) on a machine that can not communicate with the cluster nodes directly. Both the client machine and the cluster nodes can communicate with the head node that is running a scheduler.
When I perform a "Configuration Validation" by selecting:
 Parallel > Manage Configurations > Start Validation
all steps pass until the 'matlabpool' test, which fails:
 Validation Details
 Configuration: "myconfig"
 Type: ccs
 Stage: Find Resource
 Status: Succeeded
 Description:  Validation successful
 Command Line Output: (none)
 --------------------------------------
 Stage: Distributed Job
 Status: Succeeded
 Description:  Validation successful
 Command Line Output: (none)
 --------------------------------------
 Stage: Parallel Job
 Status: Succeeded
 Description:  Validation successful
 Command Line Output: (none)
 --------------------------------------
 Stage: Matlabpool
 Status: Failed
 Description:  Encountered a problem when trying to open the
 matlabpool.
 Command Line Output: (none)
 Error Report:
 Error using >
 distcomp.interactiveclient.pGetSockets>iThrowIfBadParallelJobStatus at 101
 The interactive parallel job errored with the following message:
 Lab 1 on host <cluster-node> failed to connect to the MATLAB client
 on host <client-machine>, port 27371.
 Debug Log: (none)
 --------------------------------------

Best Answer

When opening a 'matlabpool' on the client, MATLAB starts workers on the cluster that connect directly to the client session of MATLAB. As there is no direct connectivity between the client and the cluster nodes, opening the matlabpool fails.
To work around this issue, submit your MATLAB code with PARFOR statements as a matlabpool job to the cluster. In this case, there is no need to open a matlabpool on the client machine.
For example, assuming that you create a test.m file with the following MATLAB code:
function t = test()
   tStart=tic;
   x2 = [];
   n = 1000;
   parfor i = 1:n
      x2 = [x2, sin(1)];
   end
   t=toc(tStart);
end
you can submit the code to the cluster from a client machine by executing the following MATLAB code snippet in your MATLAB command window:
j = createMatlabPoolJob('configuration', 'myconfig');
createTask(j, @test, 1, {});
submit(j);
waitForState(j);
results = getAllOutputArguments(j);
results{:}
destroy(j);
The above code requires test.m to be added in the File Dependencies of the configuration.
If the MATLAB code that you want to submit is a script, i.e. does not take any arguments and does not return any values, you can use the BATCH command:
j = batch('testscript', 'configuration', 'myconfig', 'matlabpool', 2);
Related Question