MATLAB: How to replace messy data by using linear interpolation of nearby messy data

data importMATLAB

Hello!! I would like to ask for some help. I have been working with the data processing for weeks. I am quite new to this. I have been struggling to find the solution to my problem. My problem is that I have to import data from txt file which 90 % of them are alright, but some contains messy data. I will show an example of input text file in the attached file. I would like to clean this data by replacing it by the linear interpolation of the nearby existing values. My messy data is either in form of ********** or 0.
I have been seeking for an answer for a while, but no solution has been found so far.
Note that the actual input file that I work with is in format of .out not .txt but I cannot attach it here.
Thank you in advance

Best Answer

Replace all ****** values in the text file with NaN's and follow the below code: 
data = importdata('data.txt') ;
data = data.data ;
t = data(:,1) ; 
s1 = data(:,2) ; 
s2 = data(:,3) ;
%% Remove outliers in s1
stdDev = nanstd(s1) ;% Compute standard deviation

meanValue = nanmean(s1) ; % Compute mean

zFactor = 1.5; % or whatever you want.

% Create a binary map of where outliers live.

outliers = abs(s1-meanValue) > (zFactor * stdDev);
s1(outliers) = NaN ;
% GEt NaN filled 

idx = isnan(s1) ;
s1(idx) = interp1(t(~idx),s1(~idx),t(idx)) ;
%% Remove outliers in s2
stdDev = nanstd(s2) ;% Compute standard deviation
meanValue = nanmean(s2) ; % Compute mean
zFactor = 1.5; % or whatever you want.
% Create a binary map of where outliers live.
outliers = abs(s2-meanValue) > (zFactor * stdDev);
s2(outliers) = NaN ; 
% GEt NaN filled 
idx = isnan(s2) ;
s2(idx) = interp1(t(~idx),s2(~idx),t(idx)) ;
Related Question