MATLAB: How to upload 800 csv files that only contains numbers in a cell keeping their names

Question

Hello, I would like to import my 800 csv files by keeping their name so I can identify them afterwards and perform my image processing. I try this script but it takes too much time. Thank you any help is welcome.

fichiersRecherches = '*.csv'; 
[FileName,PathName] = uigetfile(fichiersRecherches,'Sélectionnez les fichiers qui ont pour extention csv', 'MultiSelect', 'on'); 
 
FileName = cellstr(FileName);
 
m = cell(1,length(FileName)); 
 
for i_file = 1:size(m,2)
    m{i_file} = xlsread(fullfile(PathName, FileName{i_file})); 
end

Answer 1

As I said: The number contain commas as decimal separators. Before such a file can be imported, in much be converted. This costs a lot of time.

Maybe this is more efficient to fix the file contents:

function Comma2Dot(FileName)
file  = memmapfile(FileName, 'writable', true);
comma = uint8(',');
point = uint8('.');
file.Data(transpose(file.Data == comma)) = point;
end

Afterwards a simple fscanf(fid, '%g;', [472, Inf]) will import the data efficiently.

By the way, all decimal places are "000000" only. This means that storing only the integer part would be ways better. Storing the data in binary format would even better again. So teh main problem is that a really inefficient file format has been chosen. and you cannot blame the import of MATLAB.

With the original data:

tic;
Comma2Dot('test2.csv');
% Emulatre DLMREAD:
fid = fopen('test2.csv');
C   = textscan(fid, '', -1, 'Delimiter', ';', 'EndOfLine', '\r\n', ...
               'CollectOutput', 1);
fclose(fid);
data = C{1};
toc
% Elapsed time is 0.120229 seconds.

Now try:

% Write data in binary format:
fid = fopen('TestData.bin', 'W');
% Number of dimensions and size
fwrite(fid, [ndims(data), size(data)], 'uint64');
fwrite(fid, data, 'uint16');
fclose(fid);
tic;
fid = fopen('TestData.bin', 'r');
nDimsData = fread(fid, 1, 'uint64');
sizeData  = fread(fid, [1, nDimsData], 'uint64');
data      = fread(fid, sizeData,  'uint16');
fclose(fid);
toc
% Elapsed time is 0.013736 seconds.

The timings might be unfair, because reading data, which have been written to disk directly before, will be taken from the disk cache. But the accelerateion is expected: Compare the file sizes of 2'632 kB for the text file and 442 kB for the binary file.

So the actual optimization is not to improve the Matlab code, but to use a smart format to store the file.