Solved – Standard error in linear regression

regressionstandard error

Lets say I run a simple linear regression. The coefficient estimate is 2, the standard error of the coefficient is 0.5 and the t value is 4.

Am I right in saying the standard error is being used in two different distributions?

The first distribution is the distribution of estimated coefficients for the population. This distribution has a mean of 2 (the population coefficient estimate) with a standard error of 0.5.

The second distribution is the distribution of t values. This distribution has a mean of 0 with a standard error of 0.5.

Best Answer

The t distribution has a mean of 0 as stated (unless using the non-central t, but that is rare in regression), but its standard deviation is based on the degrees of freedom and will not be 0.5 in this case. So the short answer to your second question is "No".

I don't think you have the 1st question correct either, but I don't fully understand your description. If we had the resources to take every possible sample of the given size from the population (possibly infinite) and find the slope/coefficient from each sample, then took the standard deviation of those estimates then we would have the standard deviation of the sampling distribution of the coefficient. Since we don't usually have the resources to take more than one sample, we can use theory, assumptions, and things we have learned from simulations along with the sample data to compute an estimate of the standard deviation of the sampling distribution of the coefficient. Since that is a bit of a mouthful we use the phrase "Standard Error of the Coefficient" as a short version of "estimated standard deviation of the sampling distribution of the coefficient". The standard error is our best estimate of how much the coefficient would change from sample to sample, it can therefore be used in hypothesis tests or confidence intervals to take into account random variation and help us make inference about the true value of the coefficient based on our estimate from the observed data.

Related Question