MATLAB: Importing large amounts of text and numerical data from csv or txt files

csvtextreadtextscan

Hi everybody

I have a very large csv or txt file with numerical as well as text data. I am looking for a way to import this data, beginning from the third row. The text strings in the data ("NA") should be represented as NaN.

The original csv data represents approx. 1000 columns and 500 rows, so specifying the handling of each column is not feasible.

Any ideas?

Thank you, best,

Chris

Best Answer

numcols = 1000;
fmt = repmat('%f',1,numcols);
fid = fopen('YourFile.csv', 'rt');
indata = textscan(fid, fmt, 'Headerlines', 2, 'Delimiter', ',', 'TreatAsEmpty', 'NA', 'CollectOutput', 1);
fclose(indata);
indata = indata{1};

Related Solutions

MATLAB: How to delete the first 2 rows (Headers) of a txt file

Well, once you've got the right content in memory, then just rewrite the file. But, specifically on that has some bearing on the content for easy ways.

But, if that is the only thing you care to do, then the simplest irrespective of what the file content is, is probably

fidi=fopen('oldfile.txt','r');
fido=fopen('newfile.txt','w');
headerlines=2;
for i=1:headerlines,fgetl(fidi);end       % skip the desired number
buf=fread(fidi,'*char');                  % suck up the rest as image
fwrite(fido,buf)                          % and spit it out verbatim
fclose('all')                             % done
%delete('oldfile.txt')                    % optionally remove the original keeping new
%movefile('newfile.txt','oldfile.txt')    % optionally rename new to old (only new left as oldfile name)

MATLAB: Delimiting a text file like “Text to columns”

%open file
fid = fopen(YourFile,'rt');
%figure out how many columns are there
firstline = fgetl(fid);
ncol = 1 + sum(firstline == '|');
%reset to beginning of file
fseek(fid,0,0);
%read data
indata = textscan(fid,repmat('%s',1,ncol),'Delimiter','|','CollectOutput',1);
%close file
fclose(fid);

The result will be indata, a cell array of strings.

Best Answer

Related Solutions

MATLAB: How to delete the first 2 rows (Headers) of a txt file

MATLAB: Delimiting a text file like “Text to columns”

Related Question