MATLAB: Neural Net – Baseline classification above chance

Question

Hello all,

I very recently started using the neural networks toolbox, and I have a basic question about the classification errors. I'm trying to sort my data into one of two categories, so baseline classification should be 50% errors. I am performing classification to predict one of 2 stimulus features at a bunch of time points. Importantly, the time points before the stimulus onset can NOT be predictive of the stimulus feature.

However, my average baseline classification errors have been only 30% in the pre-trial period ("confusion" function output "c" = .30).

1. Does this mean I should say my "empirical baseline" is 70% correct classification? 2. How could this be the case? 3. Are there better functions in the toolbox to prevent inflated estimates of classification accuracy? 4. Am I simply misunderstanding what the confusion error variable "c" means. I was undert he impression that 1-c = % correct classification (classifier accuracy).

I would appreciate any help. I realize my questions are lengthy, so I would just appreciate it if you could point me in the direction of some reading materials /resources so I can educate myself.

Thank you!

Answer 1

Changing Notation: If there are c classes with size Ni (i=1:c), N = sum(i=1,c){Ni}, then the a priori probabilities are Pi = Ni/N ;

With no other information "prior" to making calculations and/or measurements, the Naïve Bayes Classifier will assign all inputs to the class with the maximum prior probability. The corresponding per cent classifier accuracy is 100*max(Pi).

If the classes are the same size, the percent classification accuracy is 100*(1/c) (50% for c=2, 33.3% for c=3, etc)..

Hope this helps.

Thank you for formally accepting my answer

Greg

P.S. I probably have posted details in comp.ai.neural-nets and comp.soft-sys.matlab (and maybe even ANSWERS). Try searching on

 greg prior   (or priors)