MATLAB: Removing outliers in data

filteroutlier

As shown in the attached image, there is an outlier that I want to remove. I am not able to hard code it out. I have tried a few different filters including: [b,a] = butter(2, .03); filtered_theta = filtfilt(b,a,diff_theta) plot(t_position,diff_theta) hold on plot(t_position,filtered_theta) but that smooths my data too much.

Best Answer

Although picture may be worth a thousand words, having your data to work with is worth a thousand pictures.
I would begin with a Fourier transform of your data (the fft (link) function) to determine what part of your data are your signal and what part are noise.
Once you have the frequency information, and you know you can filter out the high-frequency noise, a prototype lowpass filter design for you to experiment with is:
Fs = 1000;                                                  % Sampling Frequency (Hz)
Fn = Fs/2;                                                  % Nyquist Frequency (Hz)
Wp = 100/Fn;                                                % Passband Frequency (Normalised)
Ws = 105/Fn;                                                % Stopband Frequency (Normalised)
Rp =   1;                                                   % Passband Ripple (dB)
Rs = 150;                                                   % Stopband Ripple (dB)
[n,Ws] = cheb2ord(Wp,Ws,Rp,Rs);                             % Filter Order
[z,p,k] = cheby2(n,Rs,Ws);                                  % Filter Design
[soslp,glp] = zp2sos(z,p,k);                                % Convert To Second-Order-Section For Stability
figure(3)
freqz(soslp, 2^16, Fs)                                      % Filter Bode Plot
filtered_signal = filtfilt(soslp, glp, original_signal);    % Filter Signal
This design has a much steeper transition region than your Butterworth design, so it may have less effect on the part of your signal you want to retain. Make the appropriate changes in the parameters for your signal.
Related Question