MATLAB: What are the reasons for having different outputs after several runs

bestnetDeep Learning Toolboxinitial conditionneural networkstraining

I use the same initial condition for my network, but I still get different outputs. What are the other factors that can cause this difference?

Best Answer

The short answer is that train(only if all weights are zero) or configure( anytime) assign weights depending on the state of the RNG.
If you are training in a loop over random initial weights. The best statistically unbiased choice for the best net is determined by the minimum mse of the validation set.
 rng(4151941)             % Initialize RNG with famous birthday
 for i = 1:Ntrials
   s{i} = rng;            %Save the ith state of the rng (may not need a cell) 
   net  = configure(net,x,t);
   [ net tr ] = train(net,x,t);
    mseval(i) = tr.best_vperf;  % Best mseval over all epochs of ith run
 end
 [ minmseval ibest ] = min(mseval);
 rng     = s{ibest};            % For repeating the best design
 bestnet = configure(net,x,t);
 bestIW0 = bestnet.IW
 bestb0  = bestnet.b
 bestLW0 = bestnet.LW
 [ bestnet tr ] = train(bestnet,x,t) % NO SEMICOLON TO REVEAL ALL DETAILS!!!
Hope this helps.
Thank you for formally accepting my answer
Greg
Related Question