Trend Analysis in Data Series – Which Statistical Tests to Conduct?

data visualizationregressiontrend

I have a dataset that measures students' time spent working on a set of mathematics questions. My dataframe looks a little something like this:

Participant ID Question 1 Question 2 Question 3
1107 54.2 48.9 45.0
4208 53.1 45.6 40.6

I have times for 20 questions for about 200 students. I have observed an overall decrease in time spent per question, as is shown in the figure below:

Time series of graph time spent per question

I would like to accompany this graph with a statistical measure of negative tendency.

I don't think I should use a correlation statistic as the question number is a categorical variable.

I maybe could do a OLS regression, with X being the question number and y being the time spent per question, but I am not sure how to interpret the result.

What else could I try?


Edit

Since a few people have been asking about the context in which this data was collected, you can read all about it in the study pre-registration https://osf.io/f7zgd

Best Answer

The plot itself is perhaps the best way to present the tendency.

Consider supplementing it with a robust visual indication of trend, such as a lightly colored line or curve. Building on psychometric principles (lightly and with some diffidence), I would favor an exponential curve determined by, say, the median values of the first third of the questions and the median values of the last third of the questions.

An equivalent description is to fit a straight line on a log-linear plot, as shown here.

Figure

This visualization has been engineered to support the apparent objectives of the question:

  • A title tells the reader what you want them to know.

  • The connecting line segments are visually suppressed because they are not the message.

  • The fitted line is made most prominent visually because it is the basic statistical summary -- it is the message.

  • Points that are significantly beyond the values of the fitted line (with a Bonferroni adjustment for 20 comparisons) are highlighted by making them brighter and coloring them prominently. (This assumes the vertical error bars are two-sided confidence intervals for a confidence level near 95%.)

  • The line is summarized by a single statistical measure of trend, displayed in the subtitle at the bottom: it represents an average 6.2% decrease in working time for each successive question.

This line passes through the median of the first five answer times (horizontally located at the median of the corresponding question numbers 0,1,2,3,4) and the median of the last five answer times (horizontally located at the median of the corresponding question numbers (16, 17, 18, 19, 20). This technique of using medians of the data at either extreme is advocated by John Tukey in his book EDA (Addison-Wesley 1977).

Some judgment is needed. Tukey often used the first third and last third of the data when making such exploratory fits. When I do that here, the left part of the line barely changes (it should not, since the data are consistent in that part of the plot) while the right part changes appreciably, reflecting both the greater variation in times and the greater standard errors there:

Figure 2

This time, however, (a) there are more badly fit points and (b) they consistently fall below the line. This suggests this fit does not have a sufficiently negative slope. Thus, we can have confidence that the initial exploratory estimate of $-6\%$ (or so) is one of the best possible descriptions of the trend.

Related Question