1. Coding scheme
In terms of assessing statistical significance using a t-test, it is the relative distances between the scale points that matters. Thus, (0, 0.25, 0.5, 0.75, 1) is equivalent to (1, 2, 3, 4, 5).
From my experience an equal distance coding scheme, such as those mentioned previously are the most common, and seem reasonable for Likert items.
If you explore optimal scaling, you might be able to derive an alternative coding scheme.
2. Statistical test
The question of how to assess group differences on a Likert item has already been answered here.
The first issue is whether you can link observations across the two time points. It sounds like you had a different sample.
This leads to a few options:
- Independent groups t-test: this is a simple option; it also does test for differences in group means; purists will argue that the p-value may be not entirely accurate; however, depending on your purposes, it may be adequate.
- Bootstrapped test of differences in group means: If you still want to test differences between group means but are uncomfortable with the discrete nature of dependent variable, then you could use a bootstrap to generate confidence intervals from which you could draw inferences about changes in group means.
- Mann-Whitney U test (among other non-parametric tests): Such a test does not assume normality, but it is also testing a different hypothesis.
Fixed-effects ANOVA (or its linear regression equivalent) provides a powerful family of methods to analyze these data. To illustrate, here is a dataset consistent with the plots of mean HC per evening (one plot per color):
| Color
Day | B G R | Total
-------+---------------------------------+----------
1 | 117 176 91 | 384
2 | 208 193 156 | 557
3 | 287 218 257 | 762
4 | 256 267 271 | 794
5 | 169 143 163 | 475
6 | 166 163 163 | 492
7 | 237 214 279 | 730
8 | 588 455 457 | 1,500
9 | 443 428 397 | 1,268
10 | 464 408 441 | 1,313
11 | 470 473 464 | 1,407
12 | 171 185 196 | 552
-------+---------------------------------+----------
Total | 3,576 3,323 3,335 | 10,234
ANOVA of count against day and color produces this table:
Number of obs = 36 R-squared = 0.9656
Root MSE = 31.301 Adj R-squared = 0.9454
Source | Partial SS df MS F Prob > F
-----------+----------------------------------------------------
Model | 605936.611 13 46610.5085 47.57 0.0000
|
day | 602541.222 11 54776.4747 55.91 0.0000
colorcode | 3395.38889 2 1697.69444 1.73 0.2001
|
Residual | 21554.6111 22 979.755051
-----------+----------------------------------------------------
Total | 627491.222 35 17928.3206
The model p-value of 0.0000 shows the fit is highly significant. The day p-value of 0.0000 is also highly significant: you can detect day to day changes. However, the color (semester) p-value of 0.2001 should not be considered significant: you cannot detect a systematic difference among the three semesters, even after controlling for day to day variation.
Tukey's HSD ("honest significant difference") test identifies the following significant changes (among others) in day-to-day means (regardless of semester) at the 0.05 level:
1 increases to 2, 3
3 and 4 decrease to 5
5, 6, and 7 increase to 8,9,10,11
8, 9, 10, and 11 decrease to 12.
This confirms what the eye can see in the graphs.
Because the graphs jump around quite a bit, there's no way to detect day-to-day correlations (serial correlation), which is the whole point of time series analysis. In other words, don't bother with time series techniques: there's not enough data here for them to provide any greater insight.
One should always wonder how much to believe the results of any statistical analysis. Various diagnostics for heteroscedasticity (such as the Breusch-Pagan test) don't show anything untoward. The residuals don't look very normal--they clump into some groups--so all the p-values have to be taken with a grain of salt. Nevertheless, they appear to provide reasonable guidance and help quantify the sense of the data we can get from looking at the graphs.
You can carry out a parallel analysis on the daily minima or on the daily maxima. Make sure to start with a similar plot as a guide and to check the statistical output.
Best Answer
As GaBorgulya pointed out one needs to have a model to detect the potential anomaly. This model needs to generate a "white noise" error series or be sufficient to separate signal and noise. With this model in hand based upon older data one could then compare the new value with the prediction interval. This is the classical , albeit limited approach called an "out off model test". A more comprehensive approach is to to include a "pulse variable" i.e. zeros and a 1 for the new data point and to estimate coefficients for the augmented model using all of the data. The probability of observing what you observed before you observed it ( i.e. the new value" ) is then available from the "t value" of the "pulse variable" in this augmented model. In general this approach is referred to as Intervention Detection which scans ( data mines ) the time periods to detect the points where Pulses , Level Shifts , Seasonal Pulses and Local Time Trends have been significantly evidented. In your case you are not searching for the null hypothesis but rather simply is there a potential change point at the last observation i.e. the last "1" period. Your question also suggests solutions that we have seen which detect a significant change in the mean of the last K periods alerting the analyst to the innovation.