Solved – How to use ANOVA with two or more data points per condition per participant

anovarepeated measures

I'm preparing my first ever experiment for my PhD and am currently facing some difficulties with statistics. Here is my draft experimental design:

  • 3 factors with 5, 3, 3 levels. As a result I have 45 conditions. (These numbers should be irrelevant for my question).
  • For each condition, I measure the time it takes a participant to answer a question. These questions are in a sense "generated" by the condition and are different for the different conditions.
  • This is a repeated-measures study, where all participants are measured in all conditions. So I get 45 data points per participant.

Here is the interesting bit: For each condition I can come up with multiple possible questions that test the response time in that condition. So I figured, why not ask each participant multiple questions per condition and record multiple response time values. More data points should be better after all. So I could for example get 90 data points per participant, 2 for each condition.

However, I am not sure how to handle this during the analysis:

  1. Is it a good idea to do these multiple measurements per condition and participant? Each question is rather short, so fatigue should not be an issue.
  2. How to best analyze such data? Should I just take the average of the multiple data values? Another option would be to pretend I have double the number of participants, but then a "pair" of participants is not independent.
  3. Would this have an effect on how many participants I need?

Any other advise on this issue is much appreciated.

Best Answer

  1. Yes, more data as a general rule is usually better and given that you have just one answer per condition, then the move to get 2 data points per condition for each participant is a good idea. Its not just 1 more data point, but also double the amount of data on which to model a response per person.

  2. You could take the average of the participant's answers. A strict answer to that depends on the variability between the answers. BUT, why bother, just add another factor for each participant 'question number' with two levels '1' and '2'. If there is no difference between the answer order, then the whole model will be the same as taking the mean. On the other hand, if the there is a systematic difference between the answers (for some reason), you can find out about that as well, essentially for free.

  3. 'Need' is a tricky word here, but yes, this could potentially decrease the amount of participants you need. Asking more questions increases your N, having them be within subject comparisons is even better (usually). So by asking more questions per person you should be reducing your variance, and thus increasing the likelihood of finding a significant model.

Related Question