Time Series Frequency – How to Specify Higher Frequencies Beyond Months

frequencytime series

How do we set frequency if we have data with frequency that is higher than months in a time series?

If we use months, we can set frequency to 12, but for daily data there is no obvious solution because of leap years. Rob J Hyndman suggests to use a frequency of 7, 365.25 or both for dialy time series (depending on data). Further he writes that we can set frequency to 365.25/7for weekly, 365.25/24 for hourly time series, an so on (see overview).

On the other side, a user on stackoverflow writes that "ts only works with a fixed/constant number of points per year… ts requires regularly spaced points. That is the basis of the class. You can't have different numbers of points in different years and if the series has an annual period of fluctuation it makes no sense to claim it is 7 days. Those are just the facts.". Actually the user wrote more that undermined his rejection of the use of 365.25 for daily, 365.25*24 for hourly data, and so on. Unfortunately, the comments were deleted and I can not reproduce them.

I do not know what position is correct. I guess the use of 365.25 is simply a compromise because there is not better way to model it? Or is there another way how we can set frequency? Are there arguments in favour of one of the position?

Remarks

The discussion on SO started about the R function ts but I understand the position of the SO user as a general critic about using 365.25 days although in fact a year has not that equally spaced frequency.

Rob Hyndman does mention that setting frequency with decimals will not work for some analysis because they await integers. But this is not a problem for frequency per hour, minute or seconds. Additionally, my use case is the virtualization of time series where it is okay to have decimals. I build a R package and need to know what frequency the function should expect for daily, … times series.

Best Answer

ts objects were designed for annual, quarterly and monthly series and only allow for one type of seasonality. If you must use ts objects for daily data, you have to decide which of the seasonalities is most important. Use frequency 7 if you want to model weekly seasonality and frequency 365.25 (or 365 if it must be integer) if you want to model yearly seasonality. A better solution is to use the msts class in the forecast package which allows for multiple seasonal periods. For example, with daily data you would specify the frequencies as follows:

msts(data, seasonal.periods=c(7,365.25))

But that is still rather clumsy. The best way to do it is to use a tsibble object which has an explicit time/date index column. See https://otexts.com/fpp3/tsibbles.html for an introduction.

Related Solutions

Solved – Time series forecast in R with yearly frequency

Try using the tbats() function in the forecast package for R:

x <- ts(myts, frequency=365)
fit <- tbats(x)
plot(forecast(fit))

TBATS is a generalization of ETS models designed to deal with high frequency data. See http://robjhyndman.com/papers/complex-seasonality/ for the JASA paper behind it.

Time Series Analysis – Daily Analysis Using R

Your ACF and PACF indicate that you at least have weekly seasonality, which is shown by the peaks at lags 7, 14, 21 and so forth.

You may also have yearly seasonality, although it's not obvious from your time series.

Your best bet, given potentially multiple seasonalities, may be a tbats model, which explicitly models multiple types of seasonality. Load the forecast package:

library(forecast)

Your output from str(x) indicates that x does not yet carry information about potentially having multiple seasonalities. Look at ?tbats, and compare the output of str(taylor). Assign the seasonalities:

x.msts <- msts(x,seasonal.periods=c(7,365.25))

Now you can fit a tbats model. (Be patient, this may take a while.)

model <- tbats(x.msts)

Finally, you can forecast and plot:

plot(forecast(model,h=100))

You should not use arima() or auto.arima(), since these can only handle a single type of seasonality: either weekly or yearly. Don't ask me what auto.arima() would do on your data. It may pick one of the seasonalities, or it may disregard them altogether.

EDIT to answer additional questions from a comment:

How can I check whether the data has a yearly seasonality or not? Can I create another series of total number of events per month and use its ACF to decide this?

Calculating a model on monthly data might be a possibility. Then you could, e.g., compare AICs between models with and without seasonality.

However, I'd rather use a holdout sample to assess forecasting models. Hold out the last 100 data points. Fit a model with yearly and weekly seasonality to the rest of the data (like above), then fit one with only weekly seasonality, e.g., using auto.arima() on a ts with frequency=7. Forecast using both models into the holdout period. Check which one has a lower error, using MAE, MSE or whatever is most relevant to your loss function. If there is little difference between errors, go with the simpler model; otherwise, use the one with the lower error.

The proof of the pudding is in the eating, and the proof of the time series model is in the forecasting.

To improve matters, don't use a single holdout sample (which may be misleading, given the uptick at the end of your series), but use rolling origin forecasts, which is also known as "time series cross-validation". (I very much recommend that entire free online forecasting textbook.

So Seasonal ARIMA models cannot usually handle multiple seasonalities? Is it a property of the model itself or is it just the way the functions in R are written?

Standard ARIMA models handle seasonality by seasonal differencing. For seasonal monthly data, you would not model the raw time series, but the time series of differences between March 2015 and March 2014, between February 2015 and February 2014 and so forth. (To get forecasts on the original scale, you'd of course need to undifference again.)

There is no immediately obvious way to extend this idea to multiple seasonalities.

Of course, you can do something using ARIMAX, e.g., by including monthly dummies to model the yearly seasonality, then model residuals using weekly seasonal ARIMA. If you want to do this in R, use ts(x,frequency=7), create a matrix of monthly dummies and feed that into the xreg parameter of auto.arima().

I don't recall any publication that specifically extends ARIMA to multiple seasonalities, although I'm sure somebody has done something along the lines in my previous paragraph.

Best Answer

Related Solutions

Solved – Time series forecast in R with yearly frequency

Time Series Analysis – Daily Analysis Using R

Related Question