When using libsvm, the parameter $\gamma$ is a parameter for the kernel function. Its default value is setup as $$\gamma = \frac{1}{\text{number of features.}}$$
Is there any theoretical guidance for setting up this parameter besides existing methods, e.g., grid search?
Best Answer
I would suggest the following theoretical guidance. When you are using Gaussian RBF kernel, your separating surface will be based on a combination of bell-shaped surfaces centered at each support vector. The width of each bell-shaped surface will be inversely proportional to $\gamma$. If this width is smaller than the minimum pair-wise distance for your data, you essentially have overfitting. If this width is larger than the maximum pair-wise distance for your data, all your points fall into one class and you don't have good performance either. So the optimal width should be somewhere between these two extremes.