Firstly, I want to describe the outline/idea of my code.
I am using GA to optimize a quite complex problem.
(1) I have a main funtion. Within this main function I am using GA function.
The code is like this:
[x,fval,exitflag] = ga(@(Vars)windows_objective(Vars,Case0Input_path,Engine_path,Input_path,security_count,FluidDensity,... FluidViscosity,PropDensity_0,PropDiameter_0,NoofClusters,Opt_PerfNo,Opt_PerfDia,... Opt_ClusterSpacing,Opt_ProppSize,Opt_ProppDen,Opt_MaxInjRate,Opt_MaxPropConc,Opt_RampUpSpeed,... PerfNo_Option,PerfDia_Option,ProppSize_Option,ProppDen_Option,NoofStages,... ProppPerCluster,SlurryPerCluster),6+2*NoofClusters,A,b,[],[],... lb,ub,@(Vars)windows_nlcon(Vars,NoofClusters,ProppPerCluster,SlurryPerCluster),IntCon,opts);
(2) In the GA objective function:
function [windows_objective] = windows_objective(Vars, Case0Input_path, Engine_path, Input_path, security_count, FluidDensity, FluidViscosity,PropDensity_0,PropDiameter_0,NoofClusters, Opt_PerfNo,Opt_PerfDia,Opt_ClusterSpacing,Opt_ProppSize,Opt_ProppDen,Opt_MaxInjRate,Opt_MaxPropConc,Opt_RampUpSpeed,PerfNo_Option,PerfDia_Option,ProppSize_Option,ProppDen_Option,NoofStages,ProppPerCluster,SlurryPerCluster)
an external .exe program will be called and multiple output files will be created by this external program. Therefore, within this objective function, I create a folder called "MatlabOpt" which will store all the output files created by the external program. After finish running this external program, by reading these output files, the objective function will calculate the objective function value and return it to the main function.
Whenever the objective function is called, it will check "MatlabOpt" folder like this (within the objective function):
X = sprintf('I am worker %d',labindex);disp(X)MatlabOpt_path = char(strcat(Input_path,'\MatlabOpt',num2str(labindex)));MyFolder = MatlabOpt_path; % this is a folder
if ~isdir(MyFolder) % The first time to call objective function
mkdir(MatlabOpt_path) % this is a folder copyfile(Case0Input_path, MatlabOpt_path)else rmdir(MatlabOpt_path, 's') mkdir(MatlabOpt_path) % this is a folder copyfile(Case0Input_path, MatlabOpt_path)end
Secondly, I want to describe my issues.
(1) One run of the external program will take around 40 seconds. I give the GA function population size "100" and for my problem usually GA needs to run at least 20 generations to get the optimal solution. Which means that it will take around "20*100*40=80,000s". Therefore, I want to speed up my code.
Q1: Is my undestanding/calculation of the total run time correct or not?
(2) Therefore, I want to use GA in parallel mode through the following settings:
opts = optimoptions('ga','MaxStallGenerations',10,'FunctionTolerance',1e-5,... 'MaxGenerations',200,'PlotFcn',@gaplotbestf,'PopulationSize',100,... 'InitialPopulationRange',[initiallb;initialub],... 'UseParallel',true);
(3) I am thinking about the mechanism/algorithm/idea of GA in parallel mode.
Q2: I am thinking that, for example, in my case, my poupulation size is 100, which means GA needs to calculate 100 objective function values for one generation, yes?
Q3: If I have 4 workers in parallel mode, which means one worker will calculate 100/4=25 objective function values, yes?
If I am correct, to make sure multiple workers could calculate the objective function simultaneously, within the objective function, I create the MatlabOpt folder with the worker id.
X = sprintf('I am worker %d',labindex);disp(X)MatlabOpt_path = char(strcat(Input_path,'\MatlabOpt',num2str(labindex)));
(4) When I run GA in serial, I can see that the folder is called "MatlabOpt1" and my codes could be run without problems.
(5) However, when I run GA in parallel mode, I will have the following error:
I am worker 1I am worker 1I am worker 1I am worker 1Error using windows_objective (line 16)No directories were removed.Error inwindows_frac_optimization>@(Vars)windows_objective(Vars,Case0Input_path,Engine_path,Input_path,security_count,FluidDensity,FluidViscosity,PropDensity_0,PropDiameter_0,NoofClusters,Opt_PerfNo,Opt_PerfDia,Opt_ClusterSpacing,Opt_ProppSize,Opt_ProppDen,Opt_MaxInjRate,Opt_MaxPropConc,Opt_RampUpSpeed,PerfNo_Option,PerfDia_Option,ProppSize_Option,ProppDen_Option,NoofStages,ProppPerCluster,SlurryPerCluster)(line 192) [x,fval,exitflag] = ga(@(Vars)windows_objective(Vars,Case0Input_path,Engine_path,Input_path,security_count,FluidDensity,... Error in createAnonymousFcn>@(x)fcn(x,FcnArgs{:}) (line 11)fcn_handle = @(x) fcn(x,FcnArgs{:});Error in fcnvectorizer (line 16) parfor (i = 1:popSize)Error in gaminlppenaltyfcnError in gapenaltyError in makeState (line 64) Score = FitnessFcn(state.Population(initScoreProvided+1:end,:));Error in galincon (line 17)state = makeState(GenomeLength,FitnessFcn,Iterate,output.problemtype,options);Error in gapenaltyError in gaminlpError in ga (line 393) [x,fval,exitFlag,output,population,scores] = gaminlp(FitnessFcn,nvars, ...Error in windows_frac_optimization (line 192) [x,fval,exitflag] = ga(@(Vars)windows_objective(Vars,Case0Input_path,Engine_path,Input_path,security_count,FluidDensity,... Caused by: Failure in user-supplied fitness function evaluation. GA cannot continue. Failure in initial user-supplied fitness function evaluation. GA cannot continue.
Finally, your help/comments/suggestions would be greatly appreciated!!
Thanks and best regards,
Min
Best Answer