Solved – Confidence interval for first order linear regression

confidence intervalregression

I implemented first order deming regression on an array of x and y values. I tried to calculate the confidence interval, which in my understanding should be two values (upper and lower) for slope and two values for intercept. When I plot these two values they should be two straight lines too

enter image description here

However, I have seen in few analyses showing confidence for linear regression as a curve? (example in picture below) enter image description here.

I do not understand where this comes from, what does this represents and how to implement in in a first order regression. So if anyone can guide me a place to start to understand about this would be a great help.

Best Answer

In OLS, one cannot treat the effects of the slope and intercept terms separately. A prediction $$ \hat y = \beta_0 + \beta_1 x $$ is a function of both $\beta_0$ and $\beta$ simultaneously. A proper way to find a confidence interval on this quantity is standard propagation of uncertainty. First calculate the gradient of $\hat y$ w.r.t. the two parameters: $$ \nabla_\beta \hat y = \left[ 1, x \right]^T $$ The uncertainty on $\hat y$ is given by $$ \sigma^2_{\hat y} = \left(\nabla_\beta \hat y\right)^T \Sigma_{\beta} \nabla_\beta \hat y $$ where $\Sigma_{\beta}$ is the covariance matrix of $\beta_0$ and $\beta_1$. This is equal to $$ \Sigma_\beta = \left( \begin{array}{cc} \sigma^2_0 & \rho \sigma_0 \sigma_1 \\ \rho \sigma_0 \sigma_1 & \sigma_1^2 \end{array} \right) $$ where $\rho$ is the correlation between $\beta_0$ and $\beta_1$. From here you can easily calculate the variance $\sigma^2_{\hat y}$ and set a confidence interval. You'll find that it is a quadratic function of $x$. Most statistics packages in R and Python have functions that will do this for you automatically (hence the curved lines that your question inquires about).

EDIT Regarding confidence intervals in Deming regression: this is a separate issue that my answer does not address.

Related Question