MATLAB: Reading big textfile with textscan

textscan big data

Hi,
I want to read a big textfile (~5MB) with textscan. The first 10 rows contain some informations which I don't need. Then 1024 rows with 1280 numbers per row follow. The numbers are separated by a comma. I found almost the right code except that each comma induces a new row instead separating the numbers. To use this code I have to delete the first 10 rows in my file.
fid = fopen('textfile.txt')
Out  = textscan(fid,'%f','delimiter',sprintf(','));
fclose(fid)
How should I manipulate the code so that I get a 1024×1280 array? I tried
sprintf('\n')
instead but that just gives me the first number and then stops.

Best Answer

Solution One
You do not need to delete the first ten rows from the file, because textscan has an option to ignore header lines, so you can use this instead:
Out = textscan(....,'HeaderLines',10);
To read 1280 columns of data you need to define this in the format string, so the final code could be something like this:
fmt = repmat('%f',1,1280);
fid = fopen('textfile.txt','rt')
out = textscan(fid, fmt, 'Delimiter',',', 'HeaderLines',10);
fclose(fid);
Solution Two
If the data in the matrix is all numeric (note the header lines do not need to be) then you could use the much simpler dlmread instead, something a bit like this:
out = dlmread('textfile.txt',',',10,0);
Examples
% textscan
fmt = repmat('%f',1,5);
fid = fopen('temp.txt','rt');
out_txt = textscan(fid, fmt, 'Delimiter',',', 'HeaderLines',3);
fclose(fid);
out_txt = cell2mat(out_txt)
% dlmread
out_dlm = dlmread('temp.txt',',',3,0)
display this in the command window:
out_txt =
     0     1     2     3     4
     5     6     7     8     9
out_dlm =
     0     1     2     3     4
     5     6     7     8     9
The sample text file that I used is attached here:
Related Question