MATLAB: How to choose the most significant variables from possible 57 variables for neural network input

inputinput selectionmustreadneural networkvariables

i'm going to use neural network to predict land value. i have determine the 57 variables that may affect the land value. All variables are the distance to some central business district, the distance to some school, the distance to some hospital, and the distance to the main road. How can i choose the most significant variables for the neural network input for the land value as the target.

I have been experimenting with inserting one by one variable as the neural network input to the land value as the target by using a the neural network fitting function toolbox, but none of the variables give R square higher than 50%.

So anyone can tell me what method can i use to select the most significant variables?

Best Answer

Backward stepwise (NOT stagewise) search has worked well for me in the task of classifying stars using stellar spectra( Backward stagewise search considers replacing variables that have been previously rejected).

To rank the variables:

1. Train the model using all variables

2. While the number of active variables exceeds 1

a. Find the variable that yields the best performance when it is 
   replaced by it's mean value and the model is retrained
b. Make the replacement permanent and reduce the count of active 
variables by 1.

end

Comments:

1. Sometimes I get a preliminary feel for the significant variables by using STEPWISEFIT in the backward and forward "stagewise" modes to construct 1st and 2nd order polynomial models.

2. Some variations

   a. Use a stagewise search with the NN model (Time consuming). 
   b. Use a forward search (Tends to be inferior with 
correlated variables)
   c. Either delete the variable row or fill it with zeros 
(Prolongs retraining)
   d. Replace the variable row with a random reordering 
(May need multiple (e.g., 20?) reorderings to get 
consistent results)

3. The best variation depends on the number of data points, number of variables, amount of variable correlations and complexity of the I/O relationship.

The misnamed MATLAB functions STEPWISE(GUI) and STEPWISEFIT actually do both forward and backward stagewise searches on models that are linear in their coefficients(e.g.,polynomials). Therefore they cannot be used for NN models.

However, there is a new MATLAB function, SEQUENTIALFS, that performs both forward and backward stepwise(NOT stagewise) searches on nonlinear models represented by function handles(i.e., @). Their usefullness for the NN models in the NN TBX seems to be unexplored.

Related Solutions

MATLAB: Feature selection using neural network

The most important of I inputs in "I"nput matrices with I rows and N columns depends on the "O"utput target matrix with O rows and N columns.

If input and output variables are separately standardized to zero-mean/unit-variance variables, inputs can be ranked more easily via stepwise selection algorithms. The most common of these are forward selection and backward selection. However, there are also combined forward/backward and backward/forward algorithms.

I tend to rely on the stepwise selection algorithms for linear transformations. These are available from the MATLAB function STEPWISEFIT.

A command line search yields

>> lookfor stepwise

 addedvarplot    - Create added-variable plot for stepwise regression
stepwise         - Interactive tool for stepwise regression.
stepwisefit      - Fit regression model using stepwise regression
stepwiseglm      - Create generalized linear regression model by stepwise regression.
stepwiselm       - Create 1st order linear regression model by stepwise regression.

The help and doc commands yield explanations and examples.

Another possibility is to use a 2nd order linear regression model by adding crossproducts and squares.

Most likely there are more sophisticated algorithms available. However, I have never had the need.

Hope this helps.

Thank you for formally accepting my answer

Greg

MATLAB: Neural network sensitivity analysis inputs

No. However, I have gotten very good fast suboptimal results by starting with polynomial models of standardized (zero-mean/unit-variance) variables because polynomials are linear in the coefficients. Typically, I start with backward and forward selection of linear and quadratic models. Relatively useless variables are often readily discarded at this stage before continuing with backward and sometimes forward selection with neural models.