MATLAB: How to read data (from a .dat file) seperated by lines of text into individual vectors

.datdatadata importfscanfimportloadtxt

Hi there,
I am struggling to read in a .dat file correctly. The file has a bunch of headers starting with #s at the beginning followed by data (2 columns) followed by another header and then more data etc. The data I wish to read into matlab is over 2 million lines so you can imagine that having to go back and delete these headers to load chunks of data in individually will take some time.
Is there a way for Matlab to read in a file like this and seperate the data into seperate vectors each time there's a line of text/ a gap (if I have to go through and delete all the headers). It will not 'load' the data as it contains text that over runs the amount of columns of data at the start. I have tried using the import tool in matlab and I have tried fscanf but not had much luck either.
Any advice would be much much appreciated.
Many thanks

Best Answer

S = regexp(fileread('YourFileName.dat'), '\r?\n', 'split');
if isempty(S{end}); S(end) = []; end    %regexp split leaves empty at bottom if file ended in \n which is common
nonheader = cellfun(@isempty, regexp(S, '^\s*#'));  %permit space before #
starts = strfind([false nonheader], [false true]);
stops = strfind([nonheader false], [true false]);
num_blocks = length(starts);
blocks = cell(num_blocks, 1);
for K = 1 : num_blocks
    fields = regexp(S(starts(K):stops(K)), '\s+', 'split');
    lens = cellfun(@length, fields);
    if any(lens ~= 2)
        error('data that does not have exactly two fields is present between lines %d and %d', starts(K), stops(K));
    end
    temp = vertcat(fields{:});
    blocks{K} = str2double(temp);
    end
end
Now blocks will be a cell array consisting of numeric arrays with two columns each.
Related Question