R-Squared – Why Is the Coefficient of Determination Less Than or Equal to 1?

r-squaredself-study

I have been reading about the Coefficient of Determination and am wondering why it is necessarily less than or equal to 1.

I understand that RSS is the sum of the difference between each dependent variable and it's prediction, squared.

So it makes sense that RSS will be zero if the independent variables perfectly predict the dependent ones.

I understand that TSS is the sum of the difference between the dependent variable and the mean, squared.

But why is RSS/TSS necessarily less than 1?

Best Answer

A simple thought experiment will help answer your question.

TSS is the sum of (Yi - meanY)^2.

Let us assume that we have a regression line with the value of the mean. If the mean is 5, it will simply be a horizontal line with the value of 5 throughout. And, the squared deviations from this line will be (Yi - meanY)^2 because it represents the mean. Now, we have 2 scenarios here:

The regression line that we estimate and predict is the same as this mean line. In that case, TSS = RSS because both the lines are the same. The squared deviations from the mean (Yi - meanY) and squared residuals (Yi - Yhat) will be exactly the same because the meanY and Yhat are the same.
The regression line that we estimate is different from the mean line. In that case, RSS < TSS because the model minimizes squared residuals by definition. Since the squared residuals (Yi - Yhat) are minimum by definition, they will always be less than the squared deviations from mean (Yi - meanY).

So, we only have 2 possible scenarios: either RSS = TSS or RSS < TSS. This implies that R-square will always be between 0 and 1.

Related Solutions

R – Calculating R-Squared with Centered vs. Uncentered Sums of Squares

As Stephane hinted in the comment, it is the difference between model with or without intercept that matters.

For what it is worth, here is code from one of my packages:

## cf src/library/stats/R/lm.R and case with no weights and an intercept
f <- object$fitted.values
r <- object$residuals
mss <- if (object$intercept) sum((f - mean(f))^2) else sum(f^2)
rss <- sum(r^2)
r.squared <- mss/(mss + rss)

Residuals are centered by design, leaves the fitted values with need to be centered in the intercept and not otherwise.

Solved – Can the coefficient of determination (R-squared) for a linear regression ever be zero

Yes, when ever there is no linear relationship between variables. For example, when either X or Y are constant, or where each high-low data points are balanced by high-high, or low-low data points. For example, $X=(1,1,2,2)$, $Y=(1,2,1,2)$, or $X=(-2,1,0,1,2)$, $Y=X^2$

Here are some examples: all of these have correlation of 0, and hence a coefficient of determination of zero:

No correlation graphs

It's worth noting that as soon as there's any randomness, then there's almost certainly going to be some correlation. With a small sample size, that correlation might be quite high - wouldn't be unusual to have a correlation as high as $\pm$0.3, with a sample size of 20.

Best Answer

Related Solutions

R – Calculating R-Squared with Centered vs. Uncentered Sums of Squares

Solved – Can the coefficient of determination (R-squared) for a linear regression ever be zero

Related Question