MATLAB: Does the script running “parfeval” not work when submitted as a job on an HPC

hpcjobparallelParallel Computing Toolboxparfeval

I have a script "myScript.m" that uses "parfeval" to run some functions in parallel. These functions generate files to an output directory on a network drive. When I run this script on my local machine, it works and I can see the output files on the network drive. It also works when I run an interactive MATLAB session (with a GUI) on the HPC cluster. However, it does not work when I submit a job script to our HPC job scheduler that executes:

matlab -nodisplay -r myScript

Even more puzzling is that it works if I make my code run serially. What is going on?

Best Answer

The cause of this issue is that "myScript" does not have corresponding "wait"/"fetchOutputs" calls for each "parfeval". To explain, "parfeval" takes a function and sends it to a worker to run in the background while allowing the current script to continue executing without having to wait for that function to finish. This is a form of "asynchronous computing":

https://www.mathworks.com/help/parallel-computing/asynchronous-parallel-programming.html

One should call "wait" on the "FevalFuture" object returned by "parfeval" in order to block the current script until the corresponding function running in the background finishes. Alternatively, you may use "fetchOutputs" which does the same thing except it also returns the output of the background function:

https://www.mathworks.com/help/parallel-computing/parallel.future.wait.html

https://www.mathworks.com/help/parallel-computing/parallel.future.fetchoutputs.html

Without these calls, the script you submitted to your HPC cluster "myScript" will finish before the functions in the background can finish. When this happens, your HPC job scheduler will assume your script has completed and shut down the resources allocated to your job while your background functions are still running. Thus, you do not see the expected output files.

Related Solutions

MATLAB: Don’t I receive any data from the hardware device if I try reading the data in a callback function using “parfeval”

This is an expected behavior in MATLAB. This behavior is caused because 2 different processes try to access a single Bluetooth object which is not supported by MATLAB. You are trying to use "fopen" to open a connection in MATLAB. Once this connection is opened Bluetooth object is locked by the process (process1). However, when you pass this object to "parfeval", since the "parfeval" runs this function asynchronously on different worker, it runs in a different process (process 2). When you try to read the data in process2, process2 tries to access the object but since it was locked by process1, process2 can not access it causing the issue. To resolve the issue, move the call of "fopen" in your call back function.

MATLAB: Function status in parfeval / fetchOutputs

You can check the State property of the Future object returned by parfeval to see if it has finished. You might also be able to use the fetchNext method with a timeout of 0 to collect the result if it's ready. Something like this (untested...)

while keepGoing
  fslow = parfeval(@slowFcn, ...);
  while isempty(fetchNext(fslow, 0))
    % fetchNext returns empty if it hits the timeout.
    fquick = parfeval(@quickFcn, ...);
    quickResult = fetchOutputs(fquick);
  end
  slowResult = fetchOutputs(fslow);
end

Best Answer

Related Solutions

MATLAB: Don’t I receive any data from the hardware device if I try reading the data in a callback function using “parfeval”

MATLAB: Function status in parfeval / fetchOutputs

Related Question