As the title says, is kernel regression a parametric or non-parametric method, and how can this be motivated/explained?
Regression – Is Kernel-Regression Parametric or Non-Parametric?
kernel-smoothingmathematical-statisticsnonparametricparametricregression
Best Answer
Kernel regression is considered non-parametric.
It is tempting to think of the amount of optimal smoothing as a "parameter", nevertheless on that aspect to quote from Shalizi's Advanced Data Analysis from an Elementary Point of View Chapt. 4 "Using Nonparametric Smoothing in Regression": "Strictly speaking, parameters are properties of the data-generating process alone, so the optimal amount of smoothing is not really a parameter."
Our optimal amount of smoothing depends on our smoothing method, kernel choice, but also on how much data we have. As such as the number of data is our "parameters" in a kernel regression setting, kernel regression is non-parametric in itself. This fully aligns with the definition of a non-parametric model as a model "that cannot be parametrized by a finite sample of parameters" (from Wasserman's All of Statistics Chapt. 7 "Models, Statistical Inference and Learning").