Solved – Boundary or threshold test for regression-type scatter plot

linear modelregression

I am looking for a way to test whether a boundary threshold exists in a physiological response – a sample of the data is plotted below. My hypothesis is that the X-variable imposes a physiological constraint on Y-values, thus producing a boundary 'ceiling' for maximum Y-values that decreases at higher X-values (indicated by the red line on figure). I assume any Y-values below the boundary are limited by some other factor not included in this model.

Essentially, my goal is to determine if the boundary exists and if so to derive a confidence interval for the boundary line model – similar to a linear regression model, but describing the upper bound of the Y-values, rather than the center of mass.

I'm sure something like this exists, but I haven't come across it before. Also, I would appreciate any suggestions on a better title or tags for this post – I assume there are more accurate terms for what I'm describing that would help folks find this post.

threshold

Best Answer

Such a pattern would often occur when no "boundary" actually exists.

Here I generate X and Y as independent right-skew random variates, yet such a pattern occurs:

enter image description here

The impression of any sense of a boundary in my plot is completely bogus, yet it looks very similar to yours. (There's an actual vertical boundary in this bivariate distribution at $x=80$, but I could generate very similar looking plots without any boundaries at all.)

Here's the code I used to generate the plot (in R):

x = rbeta(1000,1,10)*80
y = rbeta(1000,1,3)/1.5+.3
plot(x,y,ylim=c(0,1))

Trying it a few more times it looks like about a third of the time it gives a plot that seems to have such a slanting boundary.

No doubt a little fiddling with distributions could improve the proportion of times it occurs and at the same time make it look even more like your picture (this shifted/scaled beta(1,10)$\times$beta(1,3) was the very first counterexample I tried).

Given my picture doesn't actually have any boundary there, one should be careful of over-interpreting such a pattern. You'd need a characterization of what makes it a boundary that wouldn't generate lots of false positives on examples like the one I give.

Related Solutions

Solved – Box Cox Transforms for regression

The MASS package that comes with your R installed already, has the boxcox() function that you can use: After reading in the data, do:

    library(MASS)
    boxcox(y ~ x)

Then look at the graph this produces, which shows graphically a 95% confidence interval for the boxcox transformation parameter. But you do not really have enough data (n=10) to do this, the resulting confidence interval goes almost from -2 to 2!, with a maximum likelihood estimate of approximately 0 (a log-transform, as said before). If your real data have more observations, you should try this.

As others have said, this transformation is really trying to stabilize variances. This is not really obvious from theory, what it does, is to try to maximize a normal-distribution based likelihood function, which assumes constant variance. One could think that maximizing a normal-based likelihood would try to normalize the distribution of the residuals, but in practice the main contribution to maximizing the likelihood comes from stabilizing the variances. This is maybe not so surprising, given that the likelihood we maximizes is based on a constant variance normal distribution family!

I once wrote a slider-based demo in XLispStat, which demonstrated this clearly!

Solved – Identifying outliers based on standard error of residuals vs sample standard deviation

The authority here is just "I've been told". Despite that authority, you can think about the evidence you have: the graph itself tells you that lines at +/- 2 SE do not separate outliers from a mass of data points in the middle: they cut through zones with several data points.

It appears that your observed average score takes on distinct values 1, 2, 3, 3.5, 4, 4.5, 5. (The extra variation shown on the graph I take to be just graphical jittering.) At a guess each observed average score is just the average of two scores; otherwise the number of distinct values would be greater.

If you were to make decisions based on the SD of y alone, you would have to classify e.g. all the 1s in the same way or all the 5s in the same way. It is not clear how that you would help you in any manner whatsoever. The extreme values of 1 and 5 are not outlying in any strong or even weak sense. No value of 1 and no value of 5 is exceptional.

This graph alone doesn't suggest strong outliers, but it doesn't help much to identify them either. To check for outliers you would have to look at all the variables, e.g. in a scatter plot matrix. There is perhaps some evidence of asymmetry in that your residuals range (by eye) from almost -4 to less than 2.5, but then again that may be inevitable as it seems that most people get 4 or 5 and the model is predicting close to that for most of them.

Rather than worrying about outliers your main concern should be whether a simple linear model is the best you can do and whether there is scope for improving matters through a nonlinear model or a transformation.

A specific graphical point is that open symbols tolerate overlap better than the solid or closed symbols you are using.

Best Answer

Related Solutions

Solved – Box Cox Transforms for regression

Solved – Identifying outliers based on standard error of residuals vs sample standard deviation

Related Question