Solved – Unstandardize the slope of standardised variables in a linear regression

regressionstandardization

If I standardize my dependent and independent variable, and run a linear regression between them, the slope estimate which I have will be standardised. The variables were standardised by subtracting the mean and dividing by twice the standard deviation. I know the means and SD of the input variables

Is there any way, I can unstandardise the slope (perhaps the obvious answer is to run the regression between dependent and independent variable without standardizing them, but lets say, for the sake of argument, I cannot unstandardize my dependent and independent variables). What would be the best solution to this.

If the question is not clear, please let me know and I will try my best to restructure the question.

Thanks

Best Answer

Assuming you know, or can calculate, the standard deviations of the values of $x$ and $y$, the standardized estimate $b'$ is given by:

$$ b'=b \times \frac{s_x}{s_y} $$

So to unstandardize:

$$ b=b' \times \frac{s_y}{s_x} $$

Related Solutions

Solved – When conducting multiple regression, when should you center your predictor variables & when should you standardize them

In regression, it is often recommended to center the variables so that the predictors have mean $0$. This makes it easier to interpret the intercept term as the expected value of $Y_i$ when the predictor values are set to their means. Otherwise, the intercept is interpreted as the expected value of $Y_i$ when the predictors are set to 0, which may not be a realistic or interpretable situation (e.g. what if the predictors were height and weight?). Another practical reason for scaling in regression is when one variable has a very large scale, e.g. if you were using population size of a country as a predictor. In that case, the regression coefficients may be on a very small order of magnitude (e.g. $10^{-6}$) which can be a little annoying when you're reading computer output, so you may convert the variable to, for example, population size in millions. The convention that you standardize predictions primarily exists so that the units of the regression coefficients are the same.

As @gung alludes to and @MånsT shows explicitly (+1 to both, btw), centering/scaling does not affect your statistical inference in regression models - the estimates are adjusted appropriately and the $p$-values will be the same.

Other situations where centering and/or scaling may be useful:

when you're trying to sum or average variables that are on different scales, perhaps to create a composite score of some kind. Without scaling, it may be the case that one variable has a larger impact on the sum due purely to its scale, which may be undesirable.
To simplify calculations and notation. For example, the sample covariance matrix of a matrix of values centered by their sample means is simply $X'X$. Similarly, if a univariate random variable $X$ has been mean centered, then ${\rm var}(X) = E(X^2)$ and the variance can be estimated from a sample by looking at the sample mean of the squares of the observed values.
Related to aforementioned, PCA can only be interpreted as the singular value decomposition of a data matrix when the columns have first been centered by their means.

Note that scaling is not necessary in the last two bullet points I mentioned and centering may not be necessary in the first bullet I mentioned, so the two do not need to go hand and hand at all times.

Solved – GLMnet – “Unstandardizing” Linear Regression Coefficients

This is mostly a case of carefully working out the math. I'll handle the two predictor + intercept case, it should be clear how to generalize it.

The standardized elastic net model results in the following relationship:

$$\frac{y - \mu(y)}{\sigma(y)} = \beta_1 \frac{x_1 - \mu(x_1)}{\sigma(x_1)} + \beta_2 \frac{x_1 - \mu(x_1)}{\sigma(x_1)}$$

If you very carefully move terms around until only $y$ is on the left hand side, you'll get

$$ y = \frac{\beta_1 \sigma(y)}{\sigma(x_1)} x_1 + \frac{\beta_2 \sigma(y)}{\sigma(x_2)} x_2- \left( \frac{\beta_1 \mu(x_1)}{\sigma(x_1)} + \frac{\beta_2 \mu(x_2)}{\sigma(x_2)} \right) \sigma(y) + \mu(y) $$

which gives the relationship between the standardized and unstandardized coefficients.

Here's a quick demonstration you can test this with

X <- matrix(runif(100, 0, 1), ncol=2)
y <- 1 -2*X[,1] + X[,2]

Xst <- scale(X)
yst <- scale(y)

enet <- glmnet(X, y, lambda=0)

enetst <- glmnet(Xst, yst, lambda=0)
coef <- coefficients(enetst)

# Un-standardized betas
coef[2]*sd(y)/sd(X[,1]) # = -2
coef[3]*sd(y)/sd(X[,2]) # = 1

# Unstandardized intercept (= 1)
-(coef[2]*mean(X[,1])/sd(X[,1]) + coef[3]*mean(X[,2])/sd(X[,2]))*sd(y) + mean(y)

Best Answer

Related Solutions

Solved – When conducting multiple regression, when should you center your predictor variables & when should you standardize them

Solved – GLMnet – “Unstandardizing” Linear Regression Coefficients

Related Question