Solved – Detect sequences with specific pattern in time series data

machine learningrtime series

I am currently working on non intrusive water (consumption) monitoring. I got time series data of flow rates at a level of 0.5 Hz (1 measurement at every 2 seconds) for a time span of about half a year. Detecting single extractions like a toilet flush is quite simple. However, I also want to detect water consuming appliances like washing machines and dish washer.

Scroll down to see the question and skip the intro

Available data

Analyzing and classifying single water extractions does not work because they would be interpreted as toilet flush or tap. Therefore, I need to take the context of such extractions into consideration. I have ground truth data of several non consecutive days (e.g. 2 days in December, 2 days in February, …). Based on that ground truth data I know how a program or sequence of extractions from a washing machine usually look like (see extractions marked red int the figure below). They all have a specific start and end patterns in common (extraction of 10 seconds, short break, extraction of 100 to 150 seconds).

enter image description here

Tested approaches

I tried several approaches, to identify start or end patterns in order to detect a washing machine:

  • Dynamic Time Warping comparing a reference pattern of a washing machine with the data. No success so far due to noise and intermediate extractions in real life, like a toilet flush.
  • I tried Motif using the R-package TSMining. No success because window size is not fixed in data. Washing machines adjust their consumption based on load and program. I also thought about Motif using DTW as a kind of distance measure, but my computing power is far too low for the amount of data of let's say 20 days.
  • Currently, I am working on some kind of finite state machine. Based on water extractions and their context I try to calculate, if a washing machine is likely or not.

As mentioned before, a big issue is noise. I tried several noise reduction algorithms, but it is impossible to remove it properly. As you can see on the figure below, noise can be about +/- 2 liters/minute.

I also had a look at FFT, but I did not really know how to apply it properly due to the noise.

enter image description here

Question

How can I identify reoccurring sequences (e.g. of washing machines and dish washer) in my case? A more general approach would be preferred. So I could identify reoccurring patterns independent from specific appliances. Assigning labels to some kind of clusters of patterns would not be necessary for now.

(Keep in mind, based on household protocols about washing tasks, I could NOT identify a rythmic pattern (e.g. every 3 days at 18:00).)

Best Answer

This is similar to work identifying appliance use by monitoring electrical consumption patterns in time with one home smart meter, although they may have a little more information (e.g, phase for helping detect inductive loads as well as power draw), where you just have flow rate. They'll even have the same sorts of patterns over time for washing machines. This has been ongoing for a while, so I just did some quick Googling. These should provide help as a starting point for research in this area:
https://www.researchgate.net/publication/308764100 , https://cs.brown.edu/research/pubs/theses/masters/2011/mallya.pdf , and https://archive.uea.ac.uk/~ajb/Papers/LinesIDEAL2011.pdf
This would be easier for you if you could simultaneously access the electrical meter as well as water flows at the same time to have more features, but that's probably for a more distant future project!

Related Question