MATLAB: Neural network for classification feature extraction

classificationDeep Learning Toolboxfeature extractionmulti-classneural networkpatternnetStatistics and Machine Learning Toolbox

I have read articles about feature extraction using neural networks, my understanding is that neural networks naturally extract high-order features based on the weights on the edges of the neural networks. Is it possible to extract these features from a patternnet or other matlab neural network implementations, and then use these features for other classifiers?
In particular, I am working with about 450 training examples, 13 classes and about 280 features, and I expect some combination of features (F1-F2)*(F3-90)/(F4-10) etc, to be very predictive of my class labels, but all of the feature extraction methods I have found only work for images and not general classification problems.
Any advice is appreciated

Best Answer

The best nonlinear features are usually found from some knowledge of the physical or mathematical nature of the problem.
So, what are you trying to classify?
The next best techniques to try are linear and clustering.
 [ I N ] = size(input)       % [ 280 450 ]
 [ O N ] = size(target)      % [ 13  450 ] 
  Ntrn   = N-2*round(0.15*N  % 314
 Ntrneq  = Ntrn*O            % 4082
The largest problem appears to be the number of inputs. I would consider the following approaches.
 1. Subdivide the data into subsets of smaller input dimensions that won't choke your computer (trial and error definitely needed).
2. Standardize inputs via zscore or mapstd. Remove or modify outliers.
3. Use PLSregress to reduce input dimensions by determining effective linear combinations.
4. Consider discarding the subdominant inputs of the dominant linear combinations.
5. I'm not sure if stepwise and stepwise fit will work on classification problems with {0,1} unit vector targets. However, it is worth a try.
6. Once you have a manageable number of inputs, the use patternnet.
Hope this helps.
Thank you for formally accepting my answer
Greg
Related Question