MATLAB: Importdata can only import a few lines of a .txt file

importdataMATLAB

Hello everyone,

I am using importdata to import data from a .txt file:

filename    = '123.txt';
data        = importdata(filename);
data        = data.data;

The .txt file is as attached. I used Excel to check the .txt file and it shows 24755 lines of data in addition to 1 line of header text.
However, importdata can only import the first 66 lines.
I also tried to use the "Import Data" button on the MATLAB panel and it worked fine. Nevertheless, I need to process a large number of .txt files and I certainly need to use code instead of clicking the "Import Data" button for hundreds of times.

It seems something wrong when using this importdata function, but I could not find out why up to now. Can anyone help me figure it out?

Thanks!

Sincerely,

Junfei Tang

Best Answer

importdata() cannot handle 1.#QNAN0e+000 .

filename    = '123.txt';
opt = detectImportOptions(filename);

The above only needs to be done once for each group of files that have the same headers. After that you can loop over the files,

data = readtable(filename, opt);

This will give you a table() object that has, for example,

            time            V_Inverter_Phase_A_Inverter_Phase_B_    V_Rectifier_Phase_A_Rectifier_Phase_B_    V_Rectifier_Positive_Rectifier_Negative_    I_L_exc_    I_R_tr_1_    I_R_tr_2_
    ____________________    ____________________________________    ______________________________________    ________________________________________    ________    _________    _________
    1.39487416755957e-09                  48.60414                                0.01123047                                 -1.181915                    2.381693    -2.161491    0.5377774
    1.39488942618726e-09                  48.60414                                0.01122113                                 -1.181929                    2.381693     -2.16149    0.5379625
    1.39489002293214e-09                  48.60414                                0.01147461                                 -1.181885                    2.381693     -2.16149    0.5383048
    2.39489002240578e-09                  48.66073                                0.01171875                                 -1.181641                    2.381693    -2.111191    0.5515398
    2.51989002580944e-09                  48.66698                                0.01171875                                 -1.181641                    2.381693    -2.104903    0.5532185
    2.76989002567785e-09                  48.67606                                0.01171875                                 -1.181641                    2.381693    -2.092325    0.5566501
    3.26989002541467e-09                    48.686                                0.01184082                                 -1.181641                         NaN    -2.067164    0.5635144
    4.15460238434306e-09                  48.69257                                0.01226807                                 -1.181824                         NaN     -2.02264    0.5756616
    4.27171845118757e-09                   48.6927                                 0.0123291                                 -1.181885                    2.381692    -2.016746    0.5772697

You can refer to individual columns. Or if you want all of the columns together as a numeric array, then it would be

   data{:,:}

Related Solutions

MATLAB: How to read a CSV file with header

The better way was use the textscan function with ";" delimiter and after replace the "," by "." before the conversion of string to number.

fileID = fopen(filename,'r');
startRow = 10;
dataArray = textscan(fileID,'%s%s%s',inf,'Delimiter',';',...
    'HeaderLines', startRow-1,'ReturnOnError', false);
fclose(fileID);
dataArray{1,2} = strrep(dataArray{1,2},',','.');
dataArray{1,3} = strrep(dataArray{1,3},',','.');

MATLAB: Import data from text file

That file is not easy to read, however it’s not impossible.

It needs to be read in two stages. There might be easier ways to read it, however this works, although it may not work for all such files, unless they have exactly the same structure:

filename = 'textfile.dat';
Txt = fileread(filename);
HeaderTextEnd = strfind(Txt,'IT');
HeaderText = Txt(1:HeaderTextEnd-10);

then use one of these, depending on the MATLAB version you have:

T1 = readtable(filename, 'HeaderLines',16, 'PreserveVariableNames',1);     % With R2019b & Later
T1 = readtable(filename, 'HeaderLines',16, 'ReadVariableNames',0);         % With R2013b To R2019a

producing:

HeaderText =
    '
      *******************************************************************************
                                        <XCPLTENSOR>
                                 Isotropic exchange couplings  Jij
     
               number of sites   NQ =   8
               number of types   NT =  16
               site occupation:
                  1   2  1 1.000  2 0.000
                  2   2  3 1.000  4 0.000
                  3   2  5 1.000  6 0.000
                  4   2  7 1.000  8 0.000
                  5   2  9 0.000 10 1.000
                  6   2 11 0.000 12 1.000
                  7   2 13 1.000 14 0.000
                  8   2 15 1.000 16 0.000'

and ‘T1’ as a (98848x15) table.

I only looked at the first several rows, and they appeared to be correct. I assume everything else was imported successfully.

Note that 'PreserveVariableNames' will provide them only for information. They cannot be used to reference the variables because they are not appropriate MATLAB variable names. It will be necessary to refer to them by their column assignments instead.

Best Answer

Related Solutions

MATLAB: How to read a CSV file with header

MATLAB: Import data from text file

Related Question