I am using grid search in order to find the best values for the SVM parameters (namely C and gamma). The search is performed in two steps: coarse grid search and finer grid search, as prescribed in this "A Practical Guide to Support Vector Classification" article.
What is not clear to me is figure 2 and figure 3 in the aforementioned article: What are the name/type of these figures and how should they prepared? One thing that I am not entirely sure about is, as shown in figure 2, there are many possible pairs of (C, gamma) values that produce the same cross-validation rate (i.e. 77.5). So, based on what the best (C, gamma) in this coarse search step was determined? Is the 77.5, as I think, just a rounded number?
Also, are there other (better?) figure types that can be used to visualize the grid search steps?
Best Answer
The figures you refer to are contour plots of the hyperparameter response surface, that is the cross-validated performance for each pair of hyperparameters. Visualizing grid search is effectively only useful when you have a pair of hyperparameters, which is rarely the case.
Grid search is generally a poor approach to optimize hyperparameters, though it is still by far the most common approach. Better approaches use metaheuristic or Bayesian solvers and are available in packages like Optunity and Hyperopt .
Here are some examples of SVM hyperparameter optimization using Optunity:
Disclaimer: I'm the main developer of Optunity.
Edit: only realize now that you asked about alternative figure types instead of alternative optimization approaches, so I guess part of this answer is not directly relevant. That said, I think the suboptimality of grid search can't be stressed enough.