Solved – Anomaly detection using exponential weighted moving average

anomaly detectionexponential-smoothingmathematical-statisticsnormal distributionpredictive-models

I would like to detect anomaly using exponential weighted moving average.

I don't have series of data points. All I have is EMA(t-1) and the data point of the current time(t) DP(t).

From these data, I can calculate the new EMA(t). The EWMA constant will be 0.85 (assuming).

Now I have two EMAs={EMA(t-1) ,EMA(t)} and DP(t).

Is it possible to determine whether DPt is anomaly or not? using any logic?

For Example:

Time(t) DataPoint(t)    EMA(t-1)    EMA(t)  
1       120               0          102    
.        .                .           . 
.        .                .           . 
.        .                .           . 
.        .                .           . 
.        .                .           .     
10      300              150        277.5

I don't want to take series of data points into calculation.

At any time instance say 10, I want to know whether the data point 300 (in this case) is an anomaly or not. I also have the EMA(9)=150 and EMA(10)=277.5 for calculation (if needed)

Is there any logic to calculate this?

Update:

I thought of the below logic but not sure whether it would work for sure

ABS(DP-EMA(new))> 3 *SD {EMA(old),EMA(new)}

(or)

ABS(DP-EMA(new))> 3 *EMWSD {EMA(old),EMA(new)}

DP – Data Point
EMA – Exponential Weighted Moving Average
[1]: http://i.stack.imgur.com/5Ctwu.png

Best Answer

Why don't you keep a moving average and moving variance? See here for useful way to code them.
When you do, you can tag as anomaly anything that is, say, 3 standard deviations away from the moving average.

Notice that it's hard to tell what is an anomaly without making an assumption of what is normal i.e. what is the data generating process.

Related Question