MATLAB: Neural nets: validation vs testing

Question

wondering what the real difference is between validation and testing? both are tests of the net, it would seem

thanks

Answer 1

NOTE THE DIFFERENCE BETWEEN DESIGN AND TRAINING

1. Data Division

 DATA = TRAINING + VALIDATION + TESTING + UNSEEN

2. Design

 DATA      = DESIGN + NONDESIGN
 DESIGN    = TRAINING + VALIDATION
 NONDESIGN = TESTING + UNSEEN

3. Training

 DATA        = TRAINING + NONTRAINING
 NONTRAINING = VALIDATION + TESTING + UNSEEN

4. Training data is used to directly modify weight and bias values. Performance estimates obtained from training data are HIGHLY BIASED because the same data is directly used for both modifications and evaluation.

5. Validation data is used for

   a. ValStopping: Stops training when nontraining    
      validation error increases for m consecutive   
      epochs. This enhances the network's ability to 
      generalize to nondesign ( i.e., testing and 
      unseen) data. The MATLAB default is m = 6.
   b. Model ranking: Ranks multiple designs w.r.t. 
      performance. Performance estimates are SLIGHTLY 
      BIASED because validation data indirectly 
      affects design.

6. Testing data is used to obtain UNBIASED ESTIMATES OF NONTRAINING (including UNSEEN) DATA.

Hope this helps.

Thank you for formally accepting my answer

Greg