MATLAB: Neural networks, cross validation, seting traing,test and validation sets, all posibles subset of feature

all posibles subset of featurecross-validationneural networksseting traingtest and validation sets

Respected colleague
I want to investigate influence od different variables on my neural network model using all possible subset feature selection(there is 8191 posible subsets) I have problem to force model to use net.divideFcn with specific train, validation and test indices. I use net.divideFcn with specific indices, but my code always give me divide random functions. Its important for me to use predefined indices using divideind.In my code below i put only 50 posible cominations of variables (total numbesr of combinations is 8191) because of the speed of execution.
I dont know what is problem… I want order all variables infuence in my modelusing criteria mse or RMSE.
Thanks
clear
 load house_dataset
inputs = houseInputs;
 targets = houseTargets;
N=506;
ind=randperm(506);
index = dec2bin(1:8191);
index = index == '1';
index_transpose=transpose(index);
index_double=double(index_transpose);
results = index_double;
results(14,:) = zeros(length(results),1);
for m = 1:50
 foo = index_transpose(:,m);
inputs_i=inputs(foo,:);
k=10;
for i = 1:k
      rngstate(i) = rng;
M=50;
      valind = 1 + M*(i-1) : M*i; 
      if i==k
           tstind = 1:M;
           trnind = [ M+1:M*(k-1) , M*k+1:N ];
      else
           tstind = valind + M;
           trnind = [ 1:valind(1)-1 , tstind(end)+1:N ];
      end;
hiddenLayerSize = 3;
trnInd = ind(trnind);
valInd = ind(valind);
tstInd = ind(tstind);
Inputs_train=inputs_i(:,trnInd);
Inputs_valid=inputs_i(:,valInd);
Inputs_test=inputs_i(:,tstInd);
targets_train=targets(trnInd);
targets_valid=targets(valInd);
targets_test=targets(tstInd);
net = fitnet(hiddenLayerSize);
net.divideFcn='divideind';
net=train (fitnet(3) , Inputs_train , targets_train);
simulate=net(Inputs_test);
error=simulate-targets_test;
square_error=sum((error).^2);
RMSE(m,i)=((square_error)^0.5);
RMSE_finall=mean(RMSE,2);
end;
end;
results(14,1:50)=transpose(RMSE_finall);
results_finall=transpose(results);
best_model=sortrows(results_finall,14);

Best Answer

Ranking a large number of correlated inputs for a NN is usually a thankless task.
Since the simpler the model, the better the good inputs show up, I tend to rely on models that are linear in their coefficients (e.g., polynomials)
>> lookfor stepwise
also
 >> help sequentialfs
 >> doc sequentialfs
Hope this helps.
Thank you for formally accepting my anwser
Greg
Related Question