MATLAB: How to read non-printing characters(tab and crlf)

crlfnonprintingcharactersreadtxtdata

Hello all,

I have ill-formatted data in a text file, the values are separated by either tab or space and if the values are missing then nothing is saved(no garbage value). Following is an example of data format:

2011-06-16 19:45 \t 20.5 \t 18.7 \t 0.6 2.7

2011-06-16 20:00 \t\t 18.7 \t 0.6 \t 2.7

where \t represents a horizontal tab. Every time I have to check if the character is tab, whitespace or a value. I tried DLMREAD or TEXTSCAN but they do not solve my purpose. I have also attached a sample data file.

Thanks

Best Answer

It took some experimenting, but I can read your ‘Sample Data.txt’ file with this:

fidi = fopen('Pankaj Sample Data.txt');
d = textscan(fidi, '%10s %5s %f %f %f %f', 'Delimiter','\t', 'EndOfLine','\r\n', 'EmptyValue',NaN, 'CollectOutput',1);
d1 = d{1};      % Check Result

d2 = d{2};      % Check Result

The first 10 rows for both are:

d1 = 
      '2011-06-16'    '17:00'
      '2011-06-16'    '17:15'
      '2011-06-16'    '17:30'
      '2011-06-16'    '17:45'
      '2011-06-16'    '18:00'
      '2011-06-16'    '18:15'
      '2011-06-16'    '18:30'
      '2011-06-16'    '18:45'
      '2011-06-16'    '19:00'
      '2011-06-16'    '19:15'
d2 =
            NaN          NaN          NaN          NaN
            NaN          NaN          NaN          NaN
            NaN          NaN          NaN          NaN
            NaN          NaN          NaN          NaN
           20.1         17.4          0.3          2.6
           19.6          NaN          0.2          2.6
           20.1         17.9          0.3          2.7
           20.5          NaN          0.6          2.7
            NaN         18.7          0.6          2.7
            NaN         18.7          0.6          2.7

Related Solutions

MATLAB: Import Data From Text File and Analyse

The most efficient way to import this data is to use textscan by itself, because its options allow one to easily identify missing data (do NOT use cellfun, str2num, etc):

opt = {'MultipleDelimsAsOne',true,'TreatAsEmpty',{'__','_'}};
fid = fopen('LRFData.txt','rt');
C = textscan(fid,'%f%f',opt{:});
fclose(fid);

which imports all of the numeric data correctly, with NaN for missing data:

>> M = [C{:}]
M =
      -15.2          -15
      -14.9        -14.9
      -15.3        -15.4
        -15        -14.7
      -14.4        -14.4
      -14.2        -14.4
      -14.1        -14.1
       1.61          NaN
      -13.5        -13.6
      -13.8        -13.9
      -13.4        -13.4
      -13.6        -13.5
      -13.6        -13.5
      -13.6        -13.5
      -13.5       0.0225
        NaN          -13
      -12.9        -12.9
      -12.9        -12.7
      -12.5        -12.5
      -12.6        -12.1
      -12.4        -12.4
      -12.3        -12.3
      -12.1         79.6
        NaN        -11.7
      -11.6        -11.6
      -11.5        -11.3
      -11.2        -11.3
        -11          -11
      -10.9          -11
        -11        -10.8
      -10.6         1.04
      -12.2          NaN
      -3.37          NaN
      -10.3        -10.2
      -10.1          -10
      -9.87        -9.81
      -9.85        -9.77
      -9.71        -9.72
      -9.61        -9.43
      -9.39       0.0132
        NaN        -8.72
      -8.62        -8.51
      -8.35        -8.22
      -8.18        -8.15
      -8.11        -7.84
      -7.69        -7.64
      -7.56        -7.39
      -7.28         39.6
        NaN        -6.52
      -6.36        -6.22
      -6.07        -6.04
      -5.96        -5.82
      -5.63        -5.65
      -5.54        -5.39
      -5.26        -5.08
         -5          0.4
      -3.29          NaN
    0.00282          NaN
      -3.39          NaN
      0.126          NaN
      -0.27          NaN
        1.5          NaN
   -0.00774          NaN
       5.22         3.55
        NaN        -46.2
        NaN          7.6
        NaN       -0.925
        NaN      -0.0177
        7.7         7.76
        NaN        -85.3
        NaN         16.2
       4.67          NaN
      -1.51          NaN
    -0.0263          NaN
       -118          NaN
      -1.97          NaN
       10.4          NaN
    -0.0328          NaN
       3.65          NaN
       -141          NaN
      -2.28          NaN
    -0.0368          NaN
          4            8
        NaN         29.5
        NaN         -154
        NaN         29.6
       29.2         28.9
       28.6         29.3
       29.4         29.4
       29.2         28.4
       29.3         29.2
       29.3         28.3
       27.8        -2.42
        NaN         26.7
       26.2         26.3
       26.6         27.1
       27.2         26.8
       26.7         26.7
       26.7         26.6
         26           26
     -0.038          NaN
       26.4         26.2
       26.2         26.2
       25.8         26.3
       26.2         -2.4
        NaN     0.000995
      0.509      -0.0378
     0.0318         -154
        NaN         -154
        NaN         13.7
       13.7         13.4
       13.3         13.9
         14         14.2
       14.2         14.3
       14.3         14.6
         15         14.7
       13.8         14.4
      -2.37          NaN
       12.1         11.8
        NaN      -0.0361
        NaN         3.92
        NaN         -143
        NaN         -2.1
        NaN         7.12
       9.67          NaN
    -0.0315          NaN
       -120          NaN
       21.7          NaN
      -1.71          NaN
    -0.0244          NaN
       17.8          NaN
      -88.1          NaN
     0.0132          NaN
      -1.18          NaN
    -0.0155         11.6
       6.54         6.48
       8.01         9.51
       7.84          NaN
      -49.4          NaN
     -0.552          NaN
   -0.00529          NaN
      0.484          NaN
      -6.79          NaN
      0.116          NaN
    0.00529          NaN
       36.3          NaN
      0.773          NaN
     0.0155          NaN
      -13.4        -12.2
      -11.9        -11.6
      -11.5        -11.5
       76.6          NaN
      -11.6        -11.5
      -11.9          -12
      -12.1        -12.3
      -12.4        -12.5
      -12.6        -12.5
      -12.6        -12.7
      -12.7        -12.6
      -12.7         1.38
        NaN        -12.8
      -12.9          -13
      -13.1        -13.3
      -13.3        -13.4
      -13.4        -13.5
      -13.9          -14
      -13.9        -13.6
      -13.6       0.0244
        NaN          -14
      -14.3        -14.6
      -14.5        -14.6
      -14.6        -14.8
      -14.8          -15
      -14.9          -15
        -15        -15.2
      -15.2        -15.2
      -15.1        -15.2
      -15.3        -15.8
        -16        -15.8
      -15.6        -15.5
      -15.6        -15.5
      -15.9        -15.6
      -15.8         1.82
        NaN        -16.4
      -16.7        -17.1
      -17.3        -16.9
        -17        -17.3
      -17.5        -17.7
      -17.7        -18.1
      -18.1        -18.2
     0.0306          NaN
        -19        -19.2
      -19.3        -19.5
      -19.4        -19.3
      -19.5        -19.8
      -19.2        -19.9
        -20        -20.1
      -20.2        -20.2
        134          NaN
      -21.1        -21.2
      -21.3        -21.3
      -21.2        -21.2
      -21.6        -21.3
      -21.6        -21.5
      -22.1        -22.3
      -22.1          -22
       2.19        -25.9
        NaN        -7.47
        NaN        -23.6
      -23.8        -23.9
      -23.9        -23.8
        -24        -24.4
      -24.5        -24.7
        -25          -25
      -24.9        -25.1
     0.0357          NaN
      -26.4        -26.5
      -26.6        -26.5
      -26.5        -26.7
      -27.1        -27.4
      -26.9        -26.8
      -27.1        -27.3
      -27.1        -27.2
        151          NaN
      -28.2        -28.1
      -28.1          -28
      -28.4        -28.6
      -28.6        -28.3
      -29.1        -29.2
      -29.1        -29.1
      -28.8        -29.1
       2.39        -20.2
        NaN       0.0379
        NaN          156
        NaN        -4.12
        NaN         2.37
        NaN        -10.3
        NaN       0.0372
        NaN          -24
        -16          NaN
        149          NaN
      -23.4          NaN
       2.23          NaN
     0.0336        -14.5
      -14.4          NaN
        130          NaN
      -23.6        -6.74
        NaN          1.9
        NaN       0.0274
        NaN          102
        NaN         1.41
        NaN        -6.93
        NaN       0.0191
        NaN        -2.04
        NaN         65.1
        NaN        0.808
        NaN      0.00935
        NaN       -0.923
      -1.81          NaN
       -4.8          NaN
       23.7          NaN
      -3.63        -3.46
       -3.3        -3.14
      -3.08        -2.96
      -2.83        -2.68
      -2.49        -2.43
       -2.3        -2.18
      -1.98        -1.83
      0.148          NaN
       -0.7       -0.571
     -0.459       -0.348
     -0.237       -0.119
        NaN        0.116
      0.233         0.35
      0.464        0.568
      0.682     -0.00116
        NaN         1.73
       1.83         1.94
       2.06         2.15
        2.3         2.41
     -0.231          NaN
   0.000105       0.0557
    -0.0043      0.00376
      -18.9          NaN
      -20.3          NaN
       2.17         2.23
       2.24         2.29
       2.44         2.53
       2.62          2.7
       2.78         2.85
       2.97         3.13
       3.13            3
       3.19       -0.536
        NaN         3.12
       3.11          NaN
    -0.0117          NaN
       1.31          NaN
        -62          NaN
      -1.15          NaN
       4.11         5.64
        NaN      -0.0212
        NaN          -99
        NaN         18.7
        NaN        -1.69
        NaN      -0.0291
        NaN         22.4
        NaN         -128
        NaN       0.0201
        NaN         -2.1
        NaN      -0.0347
       26.3           15
       15.1         18.9
       22.7         18.9
        NaN         -148
        NaN        -2.35
        NaN      -0.0376
        NaN         4.09
        NaN         -155
        NaN        -2.41
        NaN      -0.0376
        NaN         -151
        NaN        -2.28
        NaN      -0.0347
        NaN         25.2
       22.8         21.9
       21.1         20.7
       20.6         -135
        NaN         19.3
         19         19.4
       19.4         19.4
       19.4         19.4
       19.4         19.4
         19         19.1
       19.1         18.9
       18.5         18.4
      -1.99          NaN
       17.3         17.3
       17.2         17.2
       17.3         17.2
       17.1           17
       16.9         17.4
       17.3           17
       16.4         16.4
    -0.0291          NaN
       15.7         15.9
       16.1         15.9
       15.8         15.6
       15.7         15.6
       15.7         15.5
       15.4         15.3
       15.4         15.2

Tested on this file:

MATLAB: Remove all NaN values from an array.

Use these lines

idx = any(isnan(D), 2);
E = D(~idx, :);

Best Answer

Related Solutions

MATLAB: Import Data From Text File and Analyse

MATLAB: Remove all NaN values from an array.

Related Question