I've been trying to detect defrost cycles using refrigeration data. I have a sensor within the fridge which simulates a product and measures the temperature every 15 minutes. Usually, based on the data, a defrost cycle occurs every 4 hours and lasts about 1:30 hours. I want to be able to detect when a defrost cycle starts and when it ends without losing any data on the way.
What I tried so far:
- Scale the data.
- Find the peaks using a threshold.
- Find the distance between two consecutive peaks.
- Calculate the median between 2 peaks.
- Find the start and the end by comparing the current value to the median, i.e. if the current value is smaller than the median but the next one is bigger, a peak starts. The opposite is valid for the end, if the current value is higher than the median but the next one is smaller, it's the end.
The algorithm works quite good if the data is not noisy. However, using the algorithm for different data sets is not an option in this case. Here is a picture of a sample data:
As you can see, there are some small peaks and valleys which are being detected as false positives and I would like to avoid that. I am looking for an approach which will eliminate the need of a threshold and does not require smoothing the data as I am interested in the data between the cycles.
Here is a sample of the data:
c(3.73709, 3.74075, 3.73709, 3.73709, 4.06058, 4.48783, 5.00664,
4.48173, 4.2742, 4.05814, 4.06424, 3.72854, 3.73709, 3.74075,
3.73709, 3.73709, 3.73709, 3.73709, 3.73709, 3.73709, 3.73709,
4.2742, 4.69535, 4.2742, 4.06058, 4.06424, 3.73709, 3.73709,
3.73709, 3.72488, 3.73709, 3.74075, 3.73709, 3.73465, 3.73709,
3.73709, 4.05814, 4.48173, 5.00664, 4.48783, 4.2742, 4.06058,
4.06424, 3.73465, 3.73709, 3.73709, 3.73709, 3.73465, 3.73465,
3.73709, 3.73709, 3.73709, 4.06058, 5.0042, 5.42779, 4.90288,
4.38163, 4.2742, 4.06058, 4.06058, 3.73709, 3.73709, 3.73709,
3.73709, 3.73709, 3.73709, 3.74075, 3.73709, 4.06058, 4.48783,
5.00664, 4.48173, 4.28031, 4.06058, 4.06424, 3.73709, 3.73709,
3.74075, 3.73709, 3.73709, 3.73709, 3.73709, 3.73709, 4.06058,
4.2742, 4.69535, 5.22393, 4.69535, 4.38163, 4.27176, 4.28031,
4.06058, 4.05814, 4.06058, 4.06058, 4.06424, 4.06058, 4.06058,
4.06058, 4.06058, 4.06058, 4.38163, 4.90288, 4.38407, 4.2742,
4.2742, 4.06424, 4.06058, 4.06424, 4.05814, 4.06058, 4.05203,
4.06058, 4.06424, 4.06424, 4.06058, 4.06424, 4.48783, 5.00664,
4.48173, 4.2742, 4.28031, 4.28031, 4.06424, 4.06424, 4.06424,
4.06058, 4.06424, 3.73709, 4.06058, 3.74075, 3.73709, 4.06424,
4.69535, 5.22393, 4.48173, 4.37797, 4.2742, 4.06424, 4.06424,
4.06424, 4.2742, 4.06058, 4.06424, 4.06424, 4.06424, 4.06424,
4.06058, 4.28031, 4.48783, 5.0042, 4.69902, 4.48173, 4.2742,
4.28397, 4.2742, 4.2742, 4.06424, 4.06058, 4.06424, 4.2742, 4.06058,
4.06058, 4.06424, 4.2742, 4.69535, 5.22393, 4.69535, 4.38163,
4.28031, 4.2742, 4.06424, 4.06058, 4.06058, 4.06424, 4.06058,
4.06424, 4.06424, 4.06058, 4.06058, 4.06058, 4.48783, 5.0042,
4.48173, 4.38163, 4.28031, 4.05814, 4.06058, 4.06058, 4.06058,
4.06058, 4.06424, 3.73465, 4.06058, 3.73465, 4.06058, 4.05814,
4.48173, 5.0042, 4.48173, 4.27176, 4.06058, 4.06058, 4.06058,
4.05814, 4.05814, 4.05814, 3.73709, 3.73709, 3.73465, 3.73709,
3.73709, 3.73465, 4.27176, 4.69291, 4.2742, 4.06058, 4.06058,
4.06058, 3.73465, 4.06058, 3.74075, 4.06058, 3.73709, 3.73709,
3.73709, 3.73465, 3.73709, 3.73465, 4.2681, 4.48173, 4.48173,
4.2742, 4.2742, 4.27176, 4.26566, 4.06058, 4.06058, 4.06058,
4.06058, 4.06058, 4.06058, 4.06424, 4.06058, 4.06058, 4.48173,
5.00664, 4.47929, 4.37797, 4.2742, 4.2742, 4.2742, 4.06058, 4.06058,
4.06058, 4.06058, 4.06058, 4.06058, 4.06058, 3.73465, 4.06058,
4.38163, 4.90043, 4.38163, 4.27176, 4.06058, 4.06058, 4.05814,
4.06058, 3.73709, 3.73709, 3.73465, 3.73465, 3.73709, 3.73709,
3.73709, 3.73709, 4.28031, 4.69535, 4.38163, 4.2742, 4.06424,
4.06424, 3.73709, 3.73465, 3.73709, 3.73709, 3.73709)
I read quite a bit of questions here but I am still confused which approach to take. Any kind of suggestions, papers, packages or approaches would be highly appreciated. Thank you very much in advance!
Kind Regards,
Boris
Best Answer
Your approach ,as you suggested, lacks generality. I took your 286 and used a piece of software that I have helped developed which is designed to aid time series analysis. Your statement suggested a periodicity of 16 ( 4 readings per hour ). H
ere is the data. AUTOBOX identified two counter-acting level shifts (visually obvious at periods 84 and 199) while forming a useful equation. 
incorporating memory , seasonal pulses and one possibly unusual data point at period 262. The Actual/Fit/Forecast is shown here with a 25 hour prediction period 
. The residuals from the model (possibly what you mean by values between the "peaks" ) are shown here 
. The ACF of the residuals suggest randomness reflecting a sufficient model 
as contrasted to the ACF of the original 286 values 
. If you have any detailed questions you can either respond here (preferably) or you can set up a chat room. Detecting when a new value is exceptional is straight forward and would be reported as a possible pulse/exceptional value since the equation can't discern based upon 1 reading whether or not it is a one-time event OR the start of a level shift OR a start of a time trend OR a start of a new seasonal pulse OR a start of a error variance change OR simply an error of measurement.
The algorithm is based upon the concept of ARIMA modelling and incorporates procedures to include deterministic effects like Pulses/Level Shifts/SEasonal Pulses and Local time Trends as espoused by Tsay and nearly everone else http://ri.search.yahoo.com/_ylt=A0LEVryKtXFVC9YAjcwPxQt.;_ylu=X3oDMTEzNDgyZmhhBGNvbG8DYmYxBHBvcwMxBHZ0aWQDRkZHRUMwXzEEc2VjA3Ny/RV=2/RE=1433544203/RO=10/RU=http%3a%2f%2fwww.unc.edu%2f~jbhill%2ftsay.pdf/RK=0/RS=H7RmXr_T9hC0Ma2RAZppd5R6TWE-
See http://www.autobox.com/cms/index.php/blog/entry/build-or-make-your-own-arima-forecasting-model for more on the suggested actual steps in model identification.