MATLAB: Puzzling parfor error during large calculation

errorparallel computingparfor

Hi there
I am running a large number of differential evolution optimization runs on a 4-parameter problem, for different input control parameters. I am using parfor to speed up things significantly. However I have experienced some weird error instances which, given that I have tested the whole thing as a normal for-loop, suggests that the parallel-toolbox is experiencing problems. The error in question is:
Caught "std::exception" Exception message is:
fl:filesystem:PathNotFound
Error using distcomp.remoteparfor/getCompleteIntervals (line 138)
An unexpected error occurred during PARFOR: Error in remote execution of
remoteParallelFunction : RUNTIME_ERROR
Error in runnested_eddy_Lee_DEparamopt_mainprog (line 23)
parfor ii=1:length(NPop)
Does anyone know what this means? I gather that a file associated with the parallel run is missing but I don't know how to fix that, if that is the case, and google gives me no clues as to the cause/solution of this problem.
Cheers
Jakob

Best Answer

I think I may have solved the issue. I dug deep into the optimization code and discovered that it saves a temporary file, and deletes it again following the calculation. Throw that into a parfor loop and the file is created and deleted by separate processes and the whole thing goes wrong. I attached a time-signature to the filename during its creation and the problem seizes to be relevant.
Related Question