MATLAB: Using importdata to import sorted files

imageryimportdatanatsortnatsortfilessort

I have a file folder that contains about 77,000 matlab files. I need to read in these files, and then in a loop, extract one value from one pixel of each (representing temperature at the selected location) to create a times series for that location. For example, I would create an array that contained the temperature for location (260,80) from all files.
The files are named "T_num2str(i)_num2str(k).mat". The first string of numbers remains the same for 1632 files, then adds one digit, stays the same for 1632 files, then adds one, etc. So for example the files are T_1160_1.mat through T_1160_1632.mat, T_1161_1.mat through T_1161_1632.mat, etc. I first tried to just copy and rename the filenames to be sequential numbers without characters, but the script was slated to take 76 days to run…I cannot rename the files without copying them, so ideally I can just use a script that will read the existing file names.
My code will successfully do this for one file, but once in a loop I am having issues with importdata saying it cannot find the file. My guess is that I am not correctly scripting the filenames. Can someone look at this code and suggest why my files may not be importing?
Note that I used the natsortfiles function because when matlab is reading the files, otherwise it is sorting:
T_1640_1.mat
T_1640_10.mat
T_1640_2.mat
Natsortfiles creates a cell of the file names. I tried using cell2mat thinking that may be the importdata issue but I was not successful. I have uploaded an example file (T_1161_1.mat)
new_path_dir='Z:\myfolder\test\'; %file directory location
filelist = dir(fullfile(new_path_dir,'*.mat')); % get list of files in directory
sorted = natsortfiles({filelist.name}); % sort file names into order
for i=1160:1208
    for k=1:numel(sorted)   
        temp=strcat(new_path_dir,'T_',num2str(i),'_',num2str(k),'.mat');
        temp2=importdata(temp);
        value = getfield(temp2,strcat('T_',num2str(i),'_',num2str(k)));
        extracted(1,k)=value(260,80);
    end
end

Best Answer

Your code is a bit mixed up, because you are confusing two different paradigms for reading multiple files:
  • reading the filenames from the OS (e.g. using dir).
  • generating the filenames from a variable (e.g. loop iterator).
https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html
Both of these might be acceptable methods for reading your files, but it makes little sense to try and combine them together. You should pick one, and stick with that. Here is a simple example of using dir, which should work for your files:
P = '.'; % directory where the files are.
D = dir(fullfile(P,'*.mat'));
N = {D(~[D.isdir]).name};
N = natsortfiles(N);
C = cell(size(N)); % preallocate the output array!
for k = 1:numel(N)
    S = load(fullfile(P,N{k}));
    [~,F] = fileparts(N{k});
    C{k} = S.(F);
end
Note how loading the field would be simpler if it had the same name in each file, because then all that would be needed would be:
       C{k} = S.fieldname;
I tested my code on the file that you uploaded and it worked as expected.
Related Question