Solved – Simple non-linear regression problem

rregression

I'm trying to model a simple use case: predicting the price of a car based on its mileage, with RStudio. I know it's a really naive model, just one variable, but it's for comprehension purposes.

My first attempt was to to use the lm function:

predictions <- lm(price~mileage, data = ads_clean)

If I plot the model using the visreg function, I get a scatter plot of my prices/mileages with a straight line (negative slope) on it. I can see according to that plot that I can obtain negative predictions (it seems normal according to the negative coefficient of the mileage).

Enter image description here

The second attempt was to elminate such negative predictions using a log10 on the price. What I'm predicting now is not the price, but the log10(price). If I want to get back to the 'right' predicted price I use 10^(predictedPrice).

predictions <- lm(log10(price)~mileage, data = ads_clean)

If I plot the model I still get a straight line on my scatter plot, but without negative predictions this time.

Enter image description here

How do I get a curve instead of a straight line? I suppose that lm can only generate straight lines (ax1 + bx2 + …. + A).

May I use another kind of function? glm?

I'd like to get such visreg (red curve):

Enter image description here

Best Answer

If you log-transformed your outcome variable and then fit a regression model, just exponentiate the predictions to plot it on the original scale.

In many cases, it's better to use some nonlinear functions such as polynomials or splines on the originale scale, as @hejseb mentioned. This post might be of interest.

Here is an example in R using the mtcars dataset. The variable used here were chosen totally arbitrarily, just for illustration purposes.

First, we plot Log(Miles/Gallon) vs. Displacement. This looks approximately linear.

Scatterplot

After fitting a linear regression model with the log-transformed Miles/Gallon, the prediction intervals on the log-scale look like this:

Fitonlogscale

Exponentiating the prediction intervals, we finally get this graphic on the original scale:

Fitonorigscale

This ensures that the prediction intervals never go below 0.

We could also fit a quadratic model on the original scale and plot the prediction intervals.

Quadraticfit

Using a quadratic fit on the original scale, we cannot be sure that the fit and prediction intervals stay above 0.

Here is the R-code that I used to generate the figures.

#------------------------------------------------------------------------------------------------------------------------------
# Load data
#------------------------------------------------------------------------------------------------------------------------------

data(mtcars)

#------------------------------------------------------------------------------------------------------------------------------
# Scatterplot with log-transformation
#------------------------------------------------------------------------------------------------------------------------------

plot(log(mpg)~disp, data = mtcars, las = 1, pch = 16, xlab = "Displacement", ylab = "Log(Miles/Gallon)")

#------------------------------------------------------------------------------------------------------------------------------
# Linear regression with log-transformation
#------------------------------------------------------------------------------------------------------------------------------

log.mod <- lm(log(mpg)~disp, data = mtcars)

#------------------------------------------------------------------------------------------------------------------------------
# Prediction intervals
#------------------------------------------------------------------------------------------------------------------------------

newframe <- data.frame(disp = seq(min(mtcars$disp), max(mtcars$disp), length = 1000))

pred <- predict(log.mod, newdata = newframe, interval = "prediction")

#------------------------------------------------------------------------------------------------------------------------------
# Plot prediction intervals on log scale
#------------------------------------------------------------------------------------------------------------------------------

plot(log(mpg)~disp
     , data = mtcars
     , ylim = c(2, 4)
     , las = 1
     , pch = 16
     , main = "Log scale"
     , xlab = "Displacement", ylab = "Log(Miles/Gallon)")

lines(pred[,"fit"]~newframe$disp, col = "steelblue", lwd = 2)
lines(pred[,"lwr"]~newframe$disp, lty = 2)
lines(pred[,"upr"]~newframe$disp, lty = 2)

#------------------------------------------------------------------------------------------------------------------------------
# Plot prediction intervals on original scale
#------------------------------------------------------------------------------------------------------------------------------

plot(mpg~disp
     , data = mtcars
     , ylim = c(8, 38)
     , las = 1
     , pch = 16
     , main = "Original scale"
     , xlab = "Displacement", ylab = "Miles/Gallon")

lines(exp(pred[,"fit"])~newframe$disp, col = "steelblue", lwd = 2)
lines(exp(pred[,"lwr"])~newframe$disp, lty = 2)
lines(exp(pred[,"upr"])~newframe$disp, lty = 2)

#------------------------------------------------------------------------------------------------------------------------------
# Quadratic regression on original scale
#------------------------------------------------------------------------------------------------------------------------------

quad.lm <- lm(mpg~poly(disp, 2), data = mtcars)

#------------------------------------------------------------------------------------------------------------------------------
# Prediction intervals
#------------------------------------------------------------------------------------------------------------------------------

newframe <- data.frame(disp = seq(min(mtcars$disp), max(mtcars$disp), length = 1000))

pred <- predict(quad.lm, newdata = newframe, interval = "prediction")

#------------------------------------------------------------------------------------------------------------------------------
# Plot prediction intervals on log scale
#------------------------------------------------------------------------------------------------------------------------------

plot(mpg~disp
     , data = mtcars
     , ylim = c(7, 36)
     , las = 1
     , pch = 16
     , main = "Original scale"
     , xlab = "Displacement", ylab = "Miles/Gallon")

lines(pred[,"fit"]~newframe$disp, col = "steelblue", lwd = 2)
lines(pred[,"lwr"]~newframe$disp, lty = 2)
lines(pred[,"upr"]~newframe$disp, lty = 2)

Related Solutions

Solved – Why use linear regression instead of average y per x

It comes down to how you would judge the quality of your model. The general approach most would agree on is that a good prediction model minimizes the unexplained portion, or errors (predicted - observed value). You could define a model that minimizes the errors overall. Or you could define a model that minimized the sum of squared errors ($\hat{\epsilon}$) overall $\sum_{i=1}^N\hat{\epsilon}_i^2 \rightarrow Minimum$. This last version is the least squares method and, if all assumptions are met, it will come up with the best linear unbiased estimator (instead of e.g. your means ratio).

Basically, taking the average of the house price by square meters will not minimize your prediction error as it can not accommodate large departures from your average house price per square meter. Only least squares, i.e. minimal sum of all squared deviations of your predicted value minus the observed values, comes up with a line that fits your data cloud best.

For a minimal example in R consider this:

hp = c(500, 750, 800, 900, 1000, 1000, 1100)
sm = c(100, 120, 130, 130, 150, 160, 165)

with house prices (hp) and square meters (sm).

When plotting, you obtain a figure where increasing sm goes hand-in-hand with increasing hp

Now, you could do what you suggested:

apsm = mean(hp/sm)

That is, you divide hp by its sm and take the average to obtain the average per squre meters (apsm).

To predict a the house price you could obtain a vector of predicted values pred ($\hat{hp}$)

pred = apsm*sm

Your predicted line now looks like this:

The problem with this line is that it is not the line that minimizes the error (hp-pred = error). Or to be more precise, it does not minimize the sum of all your squared errors.

If you were to run a linear model with eg.

lm(hp ~ sm)

your fitted line (red) would be different and it would be more efficient and unbiased:

Best Answer

Related Solutions

Solved – Why use linear regression instead of average y per x

Related Question