MATLAB: RMS error and Mean absolute error from text files

errorimage analysisImage Processing ToolboxinterpolationMATLABmean absolute errornanroot mean square error

There are 24 text files name from interpot_linear_00.txt to interpot_linear_24.txt in a folder. Each file consist on three columns( First is latitude, second is longitude and third column is temperature). I want to check best of my interpolation technique for 324 samples of temperature.
That why i randomly select 30% of samples and gives them -9999.0. Now using scatter data interpolation i consider -9999.0 as bad data values and based upon my 70% remaining samples interpolate -9999.0. After interpolating values of -9999.0 i cross check its value with its original one and calculate overall R.M.S error and Absolute mean error for each text file.
Here i am just try, but little task to remains which need assistance as in my comments
methods = {'natural'};
S = dir('interpot_linear_*.txt');
N = sort({S.name});
for K = 1 : length(N)
  infile = N{K};
  whichfile = sscanf(infile, 'interpot_linear_%c%c');
      % Load the data
      data = load(infile);
    % separate the data columns, just to make the code clear
      Lat = data(:,1); % Column 1 is Latitude
      Lon = data(:,2); % Column 2 is Longitude
      Tmp = data(:,3); % Column 3 is Temperature
     % Creating 30% of sample as -9999.0
      nr=round(n*0.3);
      TE=Tmp(randperm(n,nr));
      good_temp = find(TE);
      TE(:)=-9999.000;
      %makes another temperature column (say Tmp1) with same indices of randomly
      %seclected but -9999.0 value.
      % Find the "good" data points. It should be 70% of remaing samples?
      good_temp = find(Tmp > -9999.000);
      % find the "bad" data points
      bad_temp = find(Tmp == -9999.000);
      for midx = 1 :length(methods) 
        method = methods{midx};
        outfile = ['interpot_' method '_' whichfile '.txt'];
        % creating vector 
        T = scatteredInterpolant(Lat(good_temp), Lon(good_temp), Tmp(good_temp), method);
        % use the interpolation object to interpolate temperature values
        interp_values = T(Lat(bad_temp), Lon(bad_temp));
        % replace the bad values with the interpolated values
        Tmp(bad_temp) = interp_values;
        % Here i will calculate RMS error with formula for each file
        RMS=sqrt(mean(Tmp(:).^2 - Tmp_new(:).^2));
        %Here i am calculating Mean Absolute Error for each file with formula
        MAE=sum(abs(Tmp(:)-Tmp_new(:)))/n;
        % Here i will export results in a excel file.
      end
  end      %files
After calculating RMS error and AM error for 24 text file. I want summery statistics in excel file( File no, RMS, AME etc.) My some text files has been attached with this post.
Please help!

Best Answer

...
% Creating 30% of sample as -9999.0
nr=round(n*0.3);
TE=Tmp(randperm(n,nr));
good_temp = find(TE);
...
In the code above you've not defined n which, it appears, should be length of the vectors or
n=size(data,1);
Then randperm will produce a pseudo-random sample of those indices as you've used it. BUT, what you've called "good_temp" is that 30% subset, not the remaining 70% so you're backwards on which group is which it would appear.
BTW, using just the indices is simpler; you don't need find there...
idx30=randperm(n,nr);  % index of 30% for substitution
tmp(idx30)=nan;        % use Nan instead of "magic number" for missing value
The remaining 70% are those which aren't members of the above set. It's simpler to use logical addressing than absolute for such cases; to generate use
ibad=ismember(1:n,idx30);  % logical vector of 30% bad locations
igood=~ibad;               % complementary "good" locations vector
Using the above index arrays, you don't need to do searching for the missing values; simply use the indexing array for the subset desired in expressions or, the isnan function or the builtin functions such as nanmean and friends which automagically handle NaN as missing value (latter may require Statistics Toolbox depending on release).
Related Question