Solved – Interpreting 2-way ANOVA results

anova

Factor               F        df          p  
Gender               2.46      1       .240  
Updates              6.38      3       .008  
Gender X Updates     5.29      3       .010   
Error                        157      

Results:

My goal was to find out if there was any effect on undergraduate Eastern Washington University college students’ subjective well-being/happiness when updating their Facebook status.

I predicted that there was going to be a change on students’ subjective well-being/happiness.
Using a general linear graph to represent the the results, I found that there was an interaction between female and male participants when updating their Facebook status.

Both females’ and males’ subjective well-being/happiness increased when updating from one to three times at day; F(3,157) = 5.9, p=.01. I also found a main effect in females when doing four to six updates at day. Their subjective well-being barely decrease (M = 25, 34). Another significant effect was found on the updates overall; F(3,157) = 6.38.

I am having trouble interpreting my results found in the table.
I do not know how to integrate them and make them to make sense?

Best Answer

The significance of the main effect for updates seems to support your view that there is a relationship between the number of updates and well-being. Additionally, you didn't find any evidence of an overall difference in happiness between men and women.

A p-value of .008 means that if people with different number of updates would report being equally happy on average, you would expect to observe a sample like yours or a more extreme one (i.e. one in which apparent differences are even stronger) 0.8% of the time. This is under the conventional threshold of 5%, so you would typically conclude that there is a difference. It's difficult to describe such results simply and it's easy to misinterpret p-value, so if you are not familiar with this you should probably try to read up on it.

Beyond that, there is also apparently an interaction effect, which is a little bit trickier to interpret. It could mean that the number of updates has a stronger association with happiness for men than for women (or the other way around), that the relationship only holds for one gender but not for the other or even that the relationship goes in opposite directions depending on gender (e.g. men updating their page frequently are happier than men who don't whereas women who update are less happy than women who don't). This result does suggest that it could make sense to retain the variable in the model but did you have a reason to believe gender has an effect in the first place?

One caveat is that the number of updates is presumably not under your control. If you learned statistics from books or courses oriented toward psychology, you will often find that they use causal language to describe significant effects but this is predicated upon the fact that the data come from a randomized experiment. You can run an ANOVA on variables like gender or updating frequency but what you have is in effect a correlation, not per se evidence that updating your Facebook page changes your level of happiness. Statistically, the technique is the same but observational data like yours and experimental data afford different conclusions.

A few other thoughts:

  • When there is a significant main effect for a factor with several levels, people often run post-hoc tests to find out more about where exactly the difference lies.
  • Number of updates apparently falls in four categories (0, 1 to 3, 4 to 6 updates, more than 6 updates?). If you have the original data, categorizing it is not generally recommended, it might be better to use the counts directly.
  • Plots are very useful to interpret your results. In your situation, you might want to look at the means in each cell of the design and at boxplots.
  • You need to think about the assumptions for the test and more generally about some potential pitfalls in interpreting differences in means.
  • It's important to think about the size of the effect, as you apparently did. If the decrease you observe seems very small (you wrote that it “barely decreases”), then that's the most important conclusion, even if this difference is significant. Standardized effect sizes like eta squared provide useful information but it's perfectly fine to also look at unstandardized effect sizes (i.e. the mean difference in happiness score).
Related Question