I encountered a question while learning:
While doing a homework assignment, you fit a Linear Model to your data
set. You are thinking about changing the Linear Model to a Quadratic
one.
- Using the Quadratic Model will decrease your Irreducible Error
- (Correct answer) Using the Quadratic Model will decrease the Bias of your model
- Using the Quadratic Model will decrease the Variance of your model
- Using the Quadratic Model will decrease your Reducible Error
And an explanation for it:
Introducing the quadratic term will make your model more complicated.
More complicated models typically have lower bias at the cost of
higher variance. This has an unclear effect on Reducible Error (could
go up or down) and no effect on Irreducible Error.
I wonder what is the difference between reducing bias and reducing reducible error? I thought one always implies another.
Best Answer
Reducible error is composed of bias and variance of the estimator. While reducing the bias you are generally increasing the variance which may result in an increased reducible error.