data visualizationdescriptive statisticsregressionresidualsstandard deviation
So I understand that on the normal histogram and scatter plot, we estimate the standard deviation by looking at the portion which contains $\frac23$ of the data.
In terms of a residual plot from a regression, however, it is a randomly sized region around (x or y)= 0. How do we estimate the standard deviation of the residuals?
A dependent mixture model (hidden Markov model) may be of use, depending on the type of deviations expected.
Assume that your observations come from two distributions (or states), both of which are normally distributed, but have different mean and variance.
A number of parameters can be estimated: The initial state probabilities (2 parameters), the state transition probabilities between neighbouring data points (4 parameters) and finally the mean and variance of the two distributions (4 parameters).
In R, this model can be estimated using the depmixS4 package:
library(depmixS4)
set.seed(3)
y = rnorm(100)
y[30:35] <- rnorm(6,mean=4,sd=2)
plot(1:100,y,"l")
m <- depmix(y~1,nstates=2,ntimes=100)
fm <- fit(m)
means <- getpars(fm)[c(7,9)]
lines(1:100,means[fm@posterior$state],lwd=2,col=2)
See http://cran.r-project.org/web/packages/depmixS4/vignettes/depmixS4.pdf for references
The plots you have reproduced are generally used to detect model misspecifications, usually curvature and heteroscedasticity.
What people do, and R does automatically, is superimpose a non-parametric lowess curve on this figure. This a local regression technique that offers an informal assessment of whether higher order terms need to be included in the model, among others. You would see that if the local regression line is curved. Then you might want to examine whether your model improves upon introducing them.
Heteroscedasticity means unequal variances and as such is a violation of the OLS assumptions. Although the estimator is still consistent and unbiased, it longer is Minimum Variance (BLUE) so that's something you need to guard against. The way to judge whether your errors are heteroscedastic is by observing the scatter in your figure. Unequal scatter across the horizontal zero line can be indicative of different variances and it might be worth investigating further. There are tests for heteroscedasticity, depending on how serious you are.
When it comes to outliers, these plots do not tell us much. Sure, there seem to be some points further away from others but what we need to remember is that the residuals do not have equal variances. Thus, a better way of detecting outliers is plotting standardized residuals against fitted values, where values above three or below minus three would suggest the presence of an outlier.
Conclusions from plots can be quite subjective though and they cannot take the place of formal tests. You can find a lot more on the internet, so take a look.
Best Answer
You can do this on a plot of residuals - plain, raw residuals, not standardized or studentized - vs anything at all - residuals vs fitted, residuals vs x (indeed any predictor in a multiple regression), residuals vs index number, residuals vs a variable you didn't use in the regression, whatever you like. It's only the fact that the residuals ($y-\hat{y}$) are plotted on one of the axes that's important.
The residuals are most typically plotted on the y-axis so in that case it's the y-axis you pay attention to. It doesn't make a difference what's on the x-axis when doing this, you don't pay any attention to it. [If your residuals have been plotted on the x-axis for some reason then that is the direction you pay attention to.]
Here's a plot of residuals vs fitted values for regression on a particular data set. There are 50 observations (though one of the residuals near 0 is not visible because it lays exactly over another point). Since we want an interval containing 2/3 of the points, we'll have 1/6 of the points above the top of the interval and 1/6 of the points below it.
Since 50/6 is 8-and-a-bit, let's leave 8 points outside either end, so we'll mark our lines about halfway between the 8th and 9th points from the top and bottom of the plot. [You could mount an argument for drawing the lines right at the 9th point and saying that's 8.5 points outside each of the lines and 50-8.5-8.5 = 33 between them, but this is pretty approximate anyway so I won't worry too much. For example, 2/3 is slightly too small so I'll stick with 8 points outside each end of the interval.]
Measuring the distance between those two horizontal lines gives a distance of about 2.2 in this case. Since that interval containing the middle 2/3 of the data should be about 2 standard deviations wide, we estimate the standard deviation to be $s\approx 2.2/2 = 1.1$.
The actual standard deviation of residuals from the regression output ($\sqrt{SSE/(n-2)}\,$) was $s = 1.102$.
Not too shabby!