MATLAB: Does textscan read only 95% of the data file

textscan

My dat file has 423,000+ ascii lines that look like:
2017-08-30 12:34:56 7.89
When I use textscan, each of the 7 cells in the returned variable only have 413315 values. If I do it line by line with fgetl, I get all 423,000+ values. Textscan takes a few seconds. fgetl takes several minutes.
DATAX=textscan(fid,'%d-%d-%d %d:%d:%d %f');
Next thing to try is to sed replace '-', ':', and ' ' with \t then try again. Unfortunately I have almost 400 files like this. An opportunity to improve my shell scripting…
Any help is greatly appreciated.

Best Answer

Here's what worked:
filecontent = fileread(TheFileName);
tokens = regexp(filecontent, ...
  '(?<date>[-0-9]+\s+[0-9:.]+)\s+(?<value>-?[0-9.]+e?+)*', ...
  'names');
%dates = datetime({tokens.date});
dates = datenum({tokens.date});
values = str2double({tokens.value});
It gets both f & e formatted values. I used datenum though it seems slower than datetime so semilogy can function. Plot() works with datetime number types, but semilogy doesn't.
Related Question