Solved – Confidence interval for the slope of a GAM

generalized-additive-modelruncertainty

I need to fit a large number (thousands) of GAMs (using package mgcv in R with automated selection of smoothing terms, if that matters), and for each GAM I need to know whether the slope of the smooth term is negative over a certain range of my independent variable. Well that's easy: I just extract the fitted value at one end of the interval, and at the other end, and subtraction gives me my answer.

But does anybody have ideas for a (not-too-computationally-intensive) approach to understanding the uncertainty in this slope? The issue is that the uncertainty in the fitted values might not be independent, and I have no idea what form this dependency might generally take.

Even if you don't have ideas about how to rigorously sample from this uncertainty, I'd really appreciate any insight on whether ignoring the non-independence would be likely to yield a too-wide or too-narrow confidence interval for the slope.

Best Answer

You don't give an example so I'm not sure what you mean by an "interval" (how big?, etc), but you run the risk of having a biased estimate of the derivative of the smooth if your interval is long relative to the wiggliness of the estimated smooth.

It is quite easy to compute a finite difference-based estimate of the derivative of an estimated smooth using existing tools in mgcv. I'll summarise the approach below but this is cribbed from some blog posts I wrote that show how to do this and consider simultaneous intervals, not just the across-the-function intervals I'll show below.

I'll also use some functions I wrote to do this analysis that are in my gratia package.

library("mgcv")
library("gratia")
set.seed(2)
dat <- gamSim(1, n = 400, dist = "normal", scale = 2)
mod <- gam(y ~ s(x0) + s(x1) + s(x2) + s(x3), data = dat, method = "REML")

The basic idea is to use the type = "lpmatrix" option of the predict() method for gam objects. This returns the linear predictor matrix for a set of locations at which we evaluate the spline, which, when multiplied by the coefficients of the model yields predicted values for the smooth. We compute two of these matrices; one for a fine grid of points over the range of the covariate for the smooth, and the second at the same locations but shifted a tiny amount. We difference these two matrices to get a linear predictor matrix for the derivative at the grid of points. The variance-covariance matrix of the model coefficients is then used to created a confidence interval. As the model is additive we can ignore the other covariates and smooths and do this for each smooth in turn.

All of this is done by the fderiv() function in gratia:

## first derivatives of all smooths...
fd <- fderiv(mod)

## ...and a selected smooth
fd2 <- fderiv(mod, term = "x1")

Now fd and fd2 contain the finite difference-estimates of the derivative of all or one smooth from the fitted model. The blog posts linked below go into a lot more detail of what is going on under the hood.

We can generate a confidence interval for the derivative using the confint method

ci <- confint(fd, type = "confidence")
head(ci)

> head(ci)
  term      lower      est    upper
1   x0 -0.8496032 4.112256 9.074116
2   x0 -0.8489453 4.112287 9.073519
3   x0 -0.8448850 4.112468 9.069821
4   x0 -0.8329612 4.112988 9.058936
5   x0 -0.8108548 4.113933 9.038721
6   x0 -0.7769721 4.115360 9.007693

This has info for all the smooths, and by default is a 95% interval. First, add on a column of values where the derivatives were evaluated:

ci <- cbind(ci, x = as.vector(fd[['eval']]))

Then we can plot:

library("ggplot2")
ggplot(ci, aes(x = x, y = est, group = term)) +
  geom_ribbon(aes(ymin = lower, ymax = upper), alpha = 0.3) +
  geom_line() +
  facet_wrap( ~ term)

Giving:

Blog posts that contain more detail:

Related Solutions

R – Calculating Confidence Interval for GAM Model

In the usual way:

p <- predict(mod, newdata, type = "link", se.fit = TRUE)

Then note that p contains a component $se.fit with standard errors of the predictions for observations in newdata. You can then form CI by multipliying the SE by a value appropriate to your desired level. E.g. an approximate 95% confidence interval is formed as:

upr <- p$fit + (2 * p$se.fit)
lwr <- p$fit - (2 * p$se.fit)

You substitute in an appropriate value from a $t$ or Gaussian distribution for the interval you need.

Note that I use type = "link" as you don't say if you have a GAM or just an AM. In the GAM, you need to form the confidence interval on the scale of the linear predictor and then transform that to the scale of the response by applying the inverse of the link function:

upr <- mod$family$linkinv(upr)
lwr <- mod$family$linkinv(lwr)

Now note that these are very approximate intervals. In addition these intervals are point-wise on the predicted values and they don't take into account the fact that the smoothness selection was performed.

A simultaneous confidence interval can be computed via simulation from the posterior distribution of the parameters. I have an example of that on my blog.

If you want a confidence interval that is not conditional upon the smoothing parameters (i.e. one that takes into account that we do not know, but instead estimate, the values of the smoothness parameters), then add unconditional = TRUE to the predict() call.

Also, if you don't want to do this yourself, note that newer versions of mgcv have a plot.gam() function that returns an object with all data used to create the plots of the smooths and their confidence intervals. You can just save the output from plot.gam() in an obj

obj <- plot(model, ....)

and then inspect obj, which is a list with one component per smooth. Add seWithMean = TRUE to the plot() call to get confidence intervals that are not conditional upon smoothness parameter.

Solved – In R package mgcv, is it valid to have a random effect smooth on two continuous variables

Huh... I made my post as a guest on SO because I am still on a suspension, but then the question got migrated here!

So, if I understand you correctly, there's not really any similarity between the smooth s(x1, x2) and the random effect s(x1, x2, fac, bs = "re"), correct?

Correct. The function name "s" does not mean "smooth function" when s() is used to construct a random effect. Broadly speaking, s() is just a model term constructor routine that constructs a design matrix and a penalty matrix.

What I was envisioning was something smoothing in 2 dimensions like the former, but with some deviations from the average by factor level. You can get separate smooths per factor level using s(x1, x2, by=fac), but that completely separates the data for each factor level, rather than doing some partial pooling.

s(x1, x2, by = fac) gives you something pretty close to what you want, except that as you said, data from different factor levels are treated independently. Technically, "close" means that s(x1, x2, by = fac) gives you the correct design matrix but not the correct penalty matrix. In this regard, you are probably aiming at te(x1, x2, fac, d = c(2, 1), bs = c("tp", "re")). I have never seen such model term before, but its construction is definitely possible in mgcv:

library(mgcv)

x1 <- runif(1000)
x2 <- runif(1000)
f <- gl(5, 200)

## "smooth.spec" object
smooth_spec <- te(x1, x2, f, d = c(2, 1), bs = c("tp", "re"))

## "smooth" object
sm <- smooth.construct(smooth_spec,
                       data = list(x1 = x1, x2 = x2, f = f),
                       knots = NULL)

You can check that this smooth term has 2 smoothing parameters as expected, one for the s(x1, x2, bs = 'tp') margin, the other for the s(f, bs = 're') margin.

Specification of k turns out subtle. You need to explicitly pass nlevels(f) to the random effect margin. For example, if you want a rank-10 thin-plate regression spline,

## my example factor `f` has 5 levels
smooth_spec <- te(x1, x2, f, d = c(2, 1), bs = c("tp", "re"), k = c(10, 5))
sapply(smooth_spec$margin, "[[", "bs.dim")
# [1] 10  5

At first I was thinking that perhaps we can simply pass NA to the random effect margin, but it turns out not!

smooth_spec <- te(x1, x2, f, d = c(2, 1), bs = c("tp", "re"), k = c(10, NA))
sapply(smooth_spec$margin, "[[", "bs.dim")
# [1] 25  5  ## ?? why is it 25? something has gone wrong!

This might implies that there is a tiny bug... will have a check when available.

Best Answer

Related Solutions

R – Calculating Confidence Interval for GAM Model

Solved – In R package mgcv, is it valid to have a random effect smooth on two continuous variables

Related Question