Outliers are rare events, that do not follow the same distribution as the regular noise in your data. You can think of data with low noise, plus some outliers as a mixture distribution, thus most of the time, you get the base (small) noise, but some of the time, you get an outlier in there.
So the first curve has outliers in it, almost too many to be considered rare, but I will concede they can be called outliers. The second curve is just heavily corrupted with large noise. The outliers are the noise there. Outlier detection/correction schemes will fail on the second curve, since it is all outliers.
Next, the second curve appears to have heteroscedastic noise. The noise does not have the same noise parameters along the curve, with a standard deviation that appears to be roughly proportional to the signal. The noise also appears to come from a distribution that is not a continuous one. Only discrete levels seem to arise.
I also notice that most of your outliers seem to lie on the low side of that curve, but not all of them. A good scheme will take this fact into account.
Schemes to detect outliers perfectly will be rare and hard to come by. In fact, you will never find a perfect scheme to do so. And the latter mess of a curve you show will never be easily fit very well.
I might also question if possibly the noise might all come from ONE distribution, with very fat tails, but a lot of mass right near zero.
You will be best served if you choose a scheme that deals with the problems I mentioned above, starting with the heteroscedasticity, and the skewed distribution of the outliers, then possibly the apparently discrete nature of the noise.
Finally, what is the nature of the underlying curve here? Is it a straight line? What can we assume about that curve?
Were it my problem to solve, I think you have noise that is non-standard. A simple scheme won't be sufficient. I would be thinking along the lines of maximum likelihood estimation of a model, using a mixture distribution, thus some outliers, some just simple noise. Or I might look for a noise distribution with long tails. Again, MLE will be needed to model the process. Or, possibly you might have success by a simple linear rescaling of the entire curve before you try to model it at all.
You also need to consider how much energy you are willing to invest in solving the problem. A good solution will take some effort and time to develop. A simple scheme might work some of the time, but it won't be as good. The standard rule applies in almost anything you do: {Good, fast, cheap}. Take 2.
If you want better help, you would want to post some real data. A picture is nice, but data is better.
Best Answer