MATLAB: Import data function breaks at NaN or not a number

data importnan

I am trying to use import_data function to import dataset from the data file. This data set is generated from iterations of an experiment based upon the response from the subjects. This kind of data set is bound to end up have a Not a Number or a missing response and not extract anything after that. I am trying to figure out a way to import data and not have the function break at Not A Number or a missing response. The function extracts the row before the Not a Number and not after the NaN row. I am attaching the data file that i am interested in. Any help would be greatly appreciated.
P.S. The experiment running the experiment generates a .txt file everytime and i am not allowed to mess with that program.I am including the .txt file as well

Best Answer

With format like this you have no choice but to write your own parser. Here is a draft:
function testresult = import_testresult(filename)
    %import_testresult imports an individual test results stored in a text file
    %




    % filename:  full path of the file to import  
    % tesresult: a structure containing the test results
    filecontent =  fileread(filename);  %read whole file at once
    sections = strsplit(filecontent, ''''''''''''''''''''''''''''''''''''''''''''''''''''''''); %split into individual sections
    %the use of quotes for delimiting section is unfortunate as each quote needs to be escaped in matlab
    sections = cellfun(@(s) strsplit(s, {'\n', '\r'}), sections, 'UniformOutput', false); %split each sections into lines
    %at this point, you have 4 sections (+1 empty one for the last line of the file) split into individual lines
    %parse each section.
    %
    %section 1: assume all values are text. Note that the last cell is empty (due to the way the file is parsed earlier)
    propvalues = regexp(sections{1}(1:end-1), '\[([^\]]+)\] (.+)', 'tokens', 'once'); %extract property names and values

    propvalues = cellfun(@(pv) {matlab.lang.makeValidName(pv{1}), pv{2}}, propvalues, 'UniformOutput', false); %convert property names into valid matlab variable names

    propvalues = vertcat(propvalues{:})'; %concatenate into a single cell array

    testdescription = struct(propvalues{:}); %convert into structure

    %
    %section 2: assume all values are numbers. Note that the first and last cell are empty (due to the way the file is parsed earlier)
    propvalues = regexp(sections{2}(2:end-1), '\[([^\]]+)\] (.+)', 'tokens', 'once'); %extract property names and values
    propvalues = cellfun(@(pv) {matlab.lang.makeValidName(pv{1}), str2double(pv{2})}, propvalues, 'UniformOutput', false); %convert property names into valid matlab variable names
    propvalues = vertcat(propvalues{:})'; %concatenate into a single cell array
    testsummary = struct(propvalues{:}); %convert into structure
    %
    %section 3: assume it's a table of numbers with two header lines. missed and no response are converted to NAN
    %Note that the first and last cell are empty (due to the way the file is parsed earlier)

    results = cellfun(@(l) strsplit(l, '\t'), sections{3}(4:end-1)', 'UniformOutput', false); %split each line at the tab character
    results = str2double(vertcat(results{:})); %convert to a matrix      
    %
    %section 4: assume it's a column of number with a header.
    %Note that the first and last cell are empty (due to the way the file is parsed earlier)
    rawtimes = str2double(sections{4}(3:end-1)');
    %put everything together into one structure
    testresults = struct('description', testdescription, 'summary', testsummary, 'results', results, 'rawtimes', rawtimes);
end
There's no validation anywhere that the file conforms to the expectations, you would have to add that to make the code robust.
Related Question