In my stats class, for One-Way ANOVAs, we are taught that the F ratio represents the ratio of explained to unexplained variance.
We are also taught that $R^2$ can be interpreted as the percentage of variation in the dependent variable that is explained by the independent variable.
I have an ANOVA output that has an $R^2$ value of .09, and a F ratio of 2.6.
How is it the case that the model explains only 9% if the variance in the dependent variable, and yet there is 2 times as much explained variance than unexplained variance?
I know I must be missing something obvious. I am in a psychology class so we don't go over the math behind the scenes- just the intuition and interpretation behind the analyses.
Best Answer
An ANOVA model can be stated as follows: $$y_{ij}=\mu_i+\epsilon_{ij}$$ where $y_{ij}$ is the value of the response variable in the $j$th trial for the $i$th treatment, $i=1,\dots,r$, $j=1,\dots,n$.
At first you determine whether or not the treatment means are the same. The total variability of the $y_{ij}$ observations is measured in terms of the total deviation of each observation: $$y_{ij}-\overline{y}_{..}=(\overline{y}_{i.}-\overline{y}_{..})+(y_{ij}-\overline{y}_{i.})$$ where $\overline{y}_{..}$ is the overall mean, $\overline{y}_{i.}-\overline{y}_{..}$ is the deviation of treatment means around overall mean, and $y_{ij}-\overline{y}_{i.}$ is the deviation around treatment means. Squaring and summing you get: \begin{align*} SSTO &= \sum_i\sum_j(y_{ij}-\overline{y}_{..})^2&\text{(total sum of squares)}\\ SSTR &= \sum_i n_i(\overline{y}_{i.}-\overline{y}_{..})^2&\text{(treatment sum of squares)} \\ SSE &= \sum_i\sum_j(y_{ij}-\overline{y}_{i.})^2&\text{(error sum of squares)}\\ SSTO&=SSTR+SSE \end{align*}
$SSTO$ has $n_T-1$ degrees of freedom, where $n_T$ is the total number of observations. $SSTR$ has $r-1$ degrees of freedom, where $r$ is the number of treatment levels. $SSE$ has $n_T-r$ degrees of freedom. The $F$ ratio is: $$F^*=\frac{MSTR}{MSE},\qquad MSTR=\frac{SSTR}{r-1},MSE=\frac{SSE}{n_T-r}$$ Large values of $F^*$ support the hypothesis that not all $\mu_i$ are equal, i.e. that a significant percentage of variation is explained by the deviation of treatment means around overall mean.
If the treatments levels are quantitative independent variables, no assumption is made in analysis of variance models about the nature of the statistical relation betwem them and the response variable, but you can specify a regression function and perform a regression analysis.
In a regression analysis you are interested in a statistical relation between independent and dependent variables, not in the difference between means. So you have: \begin{align*} SSTO&=\sum_i(y_i-\overline{y})^2&\text{(total deviation)} \\ SSR&=\sum_i(\hat{y}_i-\overline{y})^2&\text{(regression sum of squares)}\\ SSE&=\sum_i(y_i-\hat{y}_i)^2&\text{(residual sum of squares)}\\ SSTO&=SSR+SSE \end{align*} where $\hat{y}_i$ is the fitted value of $y_i$, i.e. the value of $y_i$ net of the error $\epsilon_i$ (the expected value of $y_i$) according to the statistical relation you have assumed. $R$ squared is defined as: $$R^2=\frac{SSR}{SSTO}=1-\frac{SSE}{SSTO}$$ Large values of $R^2$ support the hypothesis that between independent and dependent variables there is a (linear) relation close to the one you have assumed.
An example in R.
The $F$ ratio:
supports the hypothesis that the treatments means:
are different. But $R^2$ is small:
Indeed, there is a weak linear relation between treatment levels and response:
Let me suggest Kutner, Nachtsheim, Neter, and Li, Applied Linear Statistical Models. It is a very approachable book and can also be used as a reference. Don't be frightened by the page count :)