I'm currently exploring the field of machine learning, and am pretty new to it. I'm familiar with the concept of regression and the idea of weights/coefficients for each variable to understand the importance of that variable in the model.
In the case of SVM, NN, and decision trees, is there a way to get an idea of which variables are most important? or have the most weight? Or is it not digestible in the same way that people interpret regression?
Like say I have a class I am trying to label 1 or 0 (does this person drive), and there are two other variables (the age, and the education of the person). In a regression, I can assign a coefficient to age and education to understand the predictive power of the the variables. If I do a decision tree, whatever variable I split at the top implies that that has the most importance because that divides the dataset I have by the most. But is there like some form of coefficient to that or how do I interpret a metric in the case of classifying via SVM, NN, and Decision Trees?
Also, is there a way in sklearn (python) to easily get the 'coefficients' of a classifier?
Best Answer
The answer is Yes. For almost all the models, there are parameters / coefficients / weights in there.
However, the major difference between statistical model and modern machine learning model is that machine learning models emphasize much less on variable importance, and the interpretation of the coefficient, but focusing on more for the classification accuracy.
For example, in Neural Network, for a deep structure, it is common to have hundreds thousands of coefficients. These parameters are learned from data. If we talk about MLP neural network, the coefficients are $W$ (weight) matrices for each layer. For decision tree, where to split are the parameters (for example, we have a tree "if age is smaller than 20 and gender is male", this $20$ is a parameter in the model). For support vector machine, $\alpha$ and $C$ are parameters.
Note that, these definitions are really different from regression setting, where in regression setting, usually we have one parameter for one independent variable. This is not true for machine learning models. Suppose we have 2 independent variables, neural network can still haven thousands of parameters. In SVM, number of $\alpha$ is as same as number of data points.
For python sklearn, check
get_paramsin this page for examples on SVM