MATLAB: Crossentropy loss function – What is a good performance goal

crossentropydeep learningDeep Learning Toolboxneural networksperformance goal

Good Afternoon,
Looking around ANSWER and exploring GOOGLE GROUPS i found this method by Dr. Greg Heath to define a valid training goal for the MSE performance function:
[I,N]=size(x);
[O,N]=size(t);
MSE00a=mean(var(t,0,2));                
Ntrn=floor(0.7*N);                     
Hub=floor((Ntrn-O)/(I+1+O));
MSEgoal=0.01*(Ndof/Ntrneq)*MSE00a;
And i was wondering if there is a similar method to set a Crossentropy reference goal for neural net performance, since i want to experiment different type of loss functions in order to get the best results.
King Regards,

Best Answer

These equations are not necessarily precise.
For example:
data = design + test
design = training + validation
In particular:
Test subset data should not be used to estimate design parameters.
However, since we typically let the training function randomly perform the trn/val/tst division, the separate train/val/tst subsets are not available before training.
That is why I typically design 10 nets for every trial value for the number of hidden nodes.
Hope this helps
Thank you for formally accepting my answer
Greg
Related Question