You asked for a general explanation of the concept. Your comments about the current status of forecasting at three levels of aggregation is dead on ! My answer may not precisely deal with some of your specific interests as you have focused on some distractions but I thought that I would share the follwing with you. I was asked to discuss how software I had helped write could deal with and accomodate monthly vs weekly vs daily forecasts.
My response was in three parts :
A. Overall comments on weekly versus monthly
B. The argument for parsing the momrhly forecast to dai;ly using simple ratios
C. The argument against #2 and FOR daily forecasts to be DIRECTLY developed and then used to make weekly and/or monthly forecasts.
Response A)
Monthly:
Advantages – Fast to compute, easier to model, easier to identify changes in trends, better for strategic long term forecasting
Disadvantages – If you need to plan as the daily level for capacity, people and spoilage of product then higher levels of forecasting won’t help understand the demand on a daily basis as a 1/30th ration estimate is clearly insufficient.
Causal variables that change on a frequent basis (ie daily/weekly – price, promotion) are not easily integrated into monthly analysis
Integrating Macroeconomic variables like Quarterly Unemployment requires an additional step of creating splines.
Weekly:
Advantages – When you can’t handle the modeling process at a daily level you “settle” for this. When you have very systematic cyclical cycles like “artic ice extents” that follow a rigid curve and not need for day of the week variations.
Disadvantages – Floating Holidays like Thanksgiving, Easter, Ramadan, Chinese New Year change every year and disrupt the estimate for the coefficients for the week of the year impact which CAN be handled by creating a variable for each.
The number of weeks in a year is subject to change and creates a statistical issue due to the fact that every year doesn’t have 52 weeks. We have seen the need to allocate the 53rd week to a “non-player” week to make the data a standard 52 week period which is workable, but disruptive compared to daily data.
Causal variables that change on a frequent basis (ie daily/weekly – price, promotion) are not easily integrated into monthly analysis
Integrating Macroeconomic variables like Quarterly Unemployment requires an additional step of creating splines.
Response B) ( tongue-in-cheek answer )
Assuming you had the daily data in a data warehouse and you wanted to develop daily from the monthly forecasts.
I would take monthly forecasts and partition it to daily in the following manner.
- Compute daily averages from the history database thus D1,D2,….D7 averages are known and will be used
I would compute the overall average (XBAR) and compute 7 indices I1=D1/XBAR ; I2=D2/XBAR …. I7=D7/XBAR thus the 7 I’s represent percentages i.e.
.9,1,2,…..8 for example.
I would then compute a forecast for DAY1 in the month by using the appropriate I value and get [1/30]*Monthly forecast*I , essentially adjusting the baseline daily forecast of 1/30 th of the monthly expectation.
Finally I would then normalize these DAILY forecasts so that they add to the monthly forecast.
Response C)
I should also add that the procedure I laid out in (B) is subject to a number of assumptions regarding the historical data , most of which are unrealistic in my opinion:
1) That there are no trends and no level shifts .
2) That there are no PULSES ( one time unusual values )
3) That there are no Holiday effects OR special days in the month effects OR special weeks in the month effects or beginning/end-of the month effects
4) There are no seasonality effects (monthly or weekly )
5) There have been no changes in the day-of-the-week averages over time
6) There is no autoregressive structure
7) There have been no chnages in model paramters or the error variance over time.
All of these considerations suggest that models should be developed at the daily level in order to provide information as quickly as possible.
Hope this helps !
http://autobox.com/cms/index.php/afs-university/intro-to-forecasting/doc_download/53-capabilities-presentation starting at slide 49 may be of some help to you. What you should do is totally ignore advice regarding ARIMA approaches as they are simply rear-window driving analogues and use (the front window ) causal structures such as day-of-the-week, week-of-the-month , month-of-the-year, day-of-the-month , week-of-the month etc... while incorporating any additionally identified latent variables such as level shifts, local time trends etc. Given your paucity of data some of which I mentioned is evidently inestimable but should be an inspiration for you to collect more data , possibly then identifying long-weekend effects, lead and lag effects around holidays etc. all while adjusting for anomalous data points.
As one of my gurus pointed out recently .....
"ARIMA, for example is fine for some kinds of data, okay for others and poor for still others; the same could be said of essentially anything else. There's no magic pill. – Glen_b "
Best Answer
Check out the discussion on http://www.analyticbridge.com/forum/topics/vector-time-series-analysis. The whole idea is to seamlessly integrate daily predictions and hourly predictions into one. This is done while detecting and incorporating multiple level shifts, multiple time trends , the window of response around individual holidays while incorporating weather effects, all while isolating unusual values. You could search for a mixed-frequency suite of software as you have both intra and inter day effects.