MATLAB: Using Textscan on non-uniform data

expressionnonuniformtextscan

Hello all,

I am currently trying to format outputs from a fortran code into CSV using the textscan function in Matlab. My outputs from the fortran code have a semi-uniform output, but it can change depends on the number of nodes the user requests.

In this example, the user has specified 12 nodes and the text files looks like the following:

Interpolated values at stations

168 12

867600.0000000000 % this is the time step at which the values at the following 12 nodes is found.

1 0.3170054495E+03

2 0.2983347787E+03

3 0.2857833907E+03

4 0.2825696256E+03

5 0.2795692154E+03

6 0.2806315572E+03

7 0.2811630597E+03

8 0.2814156663E+03

9 0.2814718273E+03

10 0.2811785316E+03

11 0.2807765370E+03

12 0.2798665405E+03

871200.0000000000

1 0.3042805523E+03

2 0.3033600277E+03

3 0.2913505094E+03

4 0.2790455081E+03

5 0.2709832029E+03

6 0.2680434294E+03

7 0.2677295494E+03

8 0.2684905990E+03

9 0.2690373464E+03

10 0.2696588011E+03

11 0.2699294457E+03

12 0.2697688946E+03

Currently, I have textscan skipping the first two lines. My final output goal would looks somethign like the following:

timestep 1, node 1, node 2, …, node 11, node 12

timestep 2, node 1, node 2, …, node 11, node 12

Currently, I would like the code to be smart enough to tell the number of nodes that the user supplied (provided in the second line of the above text), and also be able to distinguish between the timestep lines and the node lines.

Any suggestions?

I've attached a example of one of my text files.

Best Answer

An exercise with fscanf()

%%
ffs = "HS_full_18md_nam_outputs.txt";
fid = fopen( ffs, 'r' );
[~] = fgetl( fid );
num = fscanf( fid, '%d%d', [2,1] );
buf = fscanf( fid, ['%f', repmat('%*d%f', 1,num(2) ) ], [num(2)+1,inf] );
[~] = fclose( fid );
out = permute( buf, [2,1] );

peek on the result

>> out(1:3,1:6)
ans =
    8.676e+05   1.9204e-05    3.981e-05   5.6839e-05   7.3688e-05   9.2944e-05
    8.712e+05   2.2396e-05   4.0073e-05   5.1601e-05   6.1428e-05    7.487e-05
    8.748e+05   1.9849e-05   3.1175e-05   4.2591e-05   5.3355e-05   6.5603e-05
>>

Related Solutions

MATLAB: How do you import specific rows of numeric data (without text) for the file I have attached

Maybe not the most rubust way, but it does the job. (The numerical blocks must be followed by non-numerical text.)

%%
ffs = '1D_spectrum_1.txt'; 
fid = fopen( ffs, 'r' );
ca0 = textscan( fid, '%f%f',    'HeaderLines', 5, 'CollectOutput',true );
ca1 = textscan( fid, '%f',      'HeaderLines', 2, 'CollectOutput',true );
ca2 = textscan( fid, '%f%f%f',  'HeaderLines',12, 'CollectOutput',true );
ca3 = textscan( fid, '%f%f%f',  'HeaderLines', 1, 'CollectOutput',true );
ca4 = textscan( fid, '%f%f%f',  'HeaderLines', 1, 'CollectOutput',true );
ca5 = textscan( fid, '%f%f%f',  'HeaderLines', 1, 'CollectOutput',true );
ca6 = textscan( fid, '%f%f%f',  'HeaderLines', 1, 'CollectOutput',true );
ca7 = textscan( fid, '%f%f%f',  'HeaderLines', 1, 'CollectOutput',true );
ca8 = textscan( fid, '%f%f%f',  'HeaderLines', 1, 'CollectOutput',true );
[~] = fclose( fid );

and inspect

>> ca0{:}
ans =
      -2.3243        50.62
      -2.3147       50.609
      -2.3041       50.614
      ...
>> ca1{:}
ans =
         0.05
       0.0551
       0.0607
       0.0668
       ...
>> ca2{:}
ans =
    0.0005568         86.6         16.7
         0.15         78.1         10.8
         1.07           78         10.7
        3.711         77.9         10.7
        ...

And see textscan - Read formatted data from text file or string

In response to the first comment

Caveat: It's risky to, based on one sample file, write functions that reads mixed text files. The format of next file may be slightly different and the function fails. In the current case, may I trust that there is a letter after the single numeric field in line 5?

The documentation of textscan() is large and it takes some time to understand all its features.

Doc says:

"[...] The textscan function reapplies formatSpec throughout the entire file and stops [terminates] when it cannot match formatSpec to the data."
"[...] If you resume a text scan of a file by calling textscan with the same file identifier (fileID), then textscan automatically resumes reading at the point where it terminated the last read."
"[...] textscan skips the header lines, including the remainder of the current line."

Thus the statement

caX = textscan( fid, '%f', 'HeaderLines',4, 'CollectOutput',true );

skips 4 header lines, reads numerical fields and terminates when it encounters an item that '%f' doesn't match. The following statement

ca0 = textscan( fid, '%f%f', 'HeaderLines',1, 'CollectOutput',true );

skips the remainder of the current line, i.e the text "number of locations", reads any number of pairs of numerical fields and terminates when it encounters a non-numerical field.

To "import the number on line 5", replace the statement

ca0 = textscan( fid, '%f%f',    'HeaderLines', 5, 'CollectOutput',true );

by the two statements

caX = textscan( fid, '%f',      'HeaderLines', 4, 'CollectOutput',true );
ca0 = textscan( fid, '%f%f',    'HeaderLines', 1, 'CollectOutput',true );

It works

>> caX{:}
ans =
     6
>> ca0{:}
ans =
      -2.3243        50.62
      -2.3147       50.609
      -2.3041       50.614

MATLAB: How to extract coordinate from the header files

Think of fid as a pointer that tells textscan where to read from. It moves its way byte by byte through the file until it reaches the end. When you call textscan for Headerlin1, it's at the end of the file. Your option, then, is to either read in Headerlin1 first, and then continue with reading in the rest of the file, or use frewind to bring fid back to the beginning of the file before reading in the first line.

You can get textscan to skip the text before a number by placing that text right before the format spec: Lat:%f

Option 1

fid=fopen('aa__file002.txt');
Headerlin1 = textscan(fid,'Lat:%f Lon:%f %*[^\n]','Delimiter',','); % extra bit reads to the end of the line
A = textscan(fid,'%s %s %f','HeaderLines',5); % read file;

fclose(fid);

Option 2

fid=fopen('aa__file002.txt');
A = textscan(fid,'%s %s %f','HeaderLines',6); % read file;
frewind(fid);
Headerlin1 = textscan(fid,'Lat:%f Lon:%f','Delimiter',',');
fclose(fid);

Best Answer

Related Solutions

MATLAB: How do you import specific rows of numeric data (without text) for the file I have attached

MATLAB: How to extract coordinate from the header files

Related Question