I have used scaling on features of a model which contains 40 features (all columns are numbers) and a binary output variable.
This is the Kaggle contest here I've scaled the features assuming it would deliver better performance, but with a rbf kernel SVM, the accuracy with 10 fold CV fell from 0.92 to 0.87
Here is a box plot of features before and after scaling:
What I would like to know is why scaling decreases classifier performance? I have not seen any discussions that point at this type of outcome.
Best Answer
The problem is that you used the default parameter values in both cases. Apparently, the default values happened to be better for your data set before scaling (this is a coincidence).
When using SVM, the parameters $c$ and $\gamma$ play a crucial role and it is your task to find the best values. Your intuition is correct: the optimal performance is better when all features are scaled properly (or at least 99.99% of the time). Unfortunately, neither of your settings had optimal parameters which led to a result that seemed to reject your intuition.
Searching the optimal values for $c$ and $\gamma$ is typically done via a grid search (e.g. search a set of $<c,\gamma>$ combinations). You can estimate the performance of an SVM for a given set of parameters using cross-validation.
In pseudo-code, the general idea is this:
You can find a good beginner's tutorial here.