MATLAB: How to work with categorical variables in Neural Network Toolbox 6.0 (R2008a)

classificationDeep Learning Toolboxnominalordinal

I want to use the NEWSOM neural network for clustering. My dataset contains both numerical and categorical variables; in the documentation, however, I can only find how to work with numerical variables.

Best Answer

The ability to work with categorical variables is not available in the Neural Network Toolbox. It is supported with the Statistics Toolbox.
To work around this issue, you need to represent your categories as numerical values. In general there are two approaches:
1. Represent each category as an integer. For example if you have categories 'small', 'medium' and 'large' you could say 'small' = 0, 'medium' = 1, 'large' = 2. Although in general this idea should work fine when you only have categorical variables, you may need to pay attention to scaling when you also have numerical variables. The idea is that a numeric change for one input of the net (for example integers representing one of your categorical variables) should have roughly the same importance as the same numeric change to another input (for example one of your numerical variables). There are several tools in the Neural Network Toolbox that help you scale your inputs, see the "Processing Functions" section in the Neural Network Toolbox documentation.
Normally start with MAPMINMAX if this does not perform well you could try MAPSTD or possibly PROCESSPCA. You can apply these functions manually, or you can apply them automatically to an input by setting the 'processFcns' property for the input.
2. Use 1-of-N encoding. For example if you once again have the 'small', 'medium' and 'large' categories you could say 'small' = [1 0 0], 'medium' = [0 1 0], 'large' = [0 0 1]. When you use this method scaling should not be as much as an issue as in the other method. Also this type of encoding might work better for "unordered" categories. For small,medium,large assigning 0,1,2 in the other method seemed pretty straightforward but what if you have 'red', 'green', 'blue'? Then (for argument's sake ignoring frequency of light) there is no natural ordering so it is difficult to say whether 'red' = 0, 'green' = 1, 'blue' = 2 is better (will work better) than 'red' = 2, 'green' = 1, 'blue' = 0. The 1-of-N encoding 'red' = [1 0 0], 'green' = [0 1 0], 'blue'= [0 0 1], should not have this problem.
Related Question