MATLAB: Missing data, time series

MATLABmissing datatime series

I am recording groundwater level at hourly intervalls using a data logger. I have data from 31 May 2017 09:00 hrs until 3 May 2019 03:00 hrs.

I noticed there are missing data as the total number of records should be 16843.

I generate a timeseries bb=[a1:ap:ae]; covering the indicated period where a1=datenum(2017,5,31,9,0,0), ap=datenum(0,0,0,1,0,0) and ae=datenum(2019,5,3,3,0,0) which gives the length of the time series being 16843

I read the logger data from excel into matlab with date in first column aa(:,1) while the second column (aa(:,2) contains the recorded logger data. I convert the Excel data to matlab data.

How can I now merge the data from aa with the dates in bb so that a second column (bb(:,2) has the reccorded logger data merged/combined with the correct date.

thanks before hand

Johannes

Best Answer

All you need to do is call retime and tell it what you want the new time v ecgtor to be. Sometimes it's as simple as 'hourly', other times you'll need to create a time vector yourself. It sounds like you want every hour represented. 'hourly' does that.

Related Solutions

MATLAB: Convert hourly temperature data into daily in the specific time interval

If this is in a datetime variable type, how about you offset your original time by 1 hr (e.g., yourVarName - hours(1)). After that you can group them by date and perform average over each group (consider using findgroups function). This can be done easily using splitapply function if this is a table variable.

MATLAB: Create a single average day out of hourly data (33days)

"First row is date second is hourly mean temp."

That would be column, not row, as just a nit.

Yes, either groupsummary or varfun will do the job, but you need to create the grouping variable to use them...beginning with a timetable, tt that looks like the above, then

>> tt.Day=day(tt.Date);
>> varfun(@mean,tt,'groupingvariables','Day')
ans =
  2×3 timetable
            Date            Day    GroupCount    mean_Temp
    ____________________    ___    __________    _________
    07-Mar-2019 11:00:00     7         13         15.692  
    08-Mar-2019 00:00:00     8          8           13.5  
>> groupsummary(tt,'Day','mean')
ans =
  2×3 table
    Day    GroupCount    mean_Temp
    ___    __________    _________
     7         13         15.692  
     8          8           13.5  
>> [g,ig]=findgroups(tt.Day);
>> splitapply(@mean,tt.Temp,g)
ans =
   15.6923
   13.5000
>> table(tt.Date(ig),splitapply(@mean,tt.Temp,g))
ans =
  2×2 table
            Var1             Var2 
    ____________________    ______
    07-Mar-2019 17:00:00    15.692
    07-Mar-2019 18:00:00      13.5
>>

are various routes to the end...

>> retime(timetable(tt.Date(ig),splitapply(@mean,tt.Temp,g),'VariableNames',{'MeanTemp'}),'daily','nearest')
ans =
  2×1 timetable
            Time            MeanTemp
    ____________________    ________
    07-Mar-2019 00:00:00     15.692 
    08-Mar-2019 00:00:00       13.5 
>>

also gets rid of the odd beginning hour for each day if your data aren't all collected beginning at 00:00 each day.

You could add some error checking to ensure that don't include days that are just nightime or daytime measurements so those would bias results where there is some extreme from day to night.

ERRATUM:

Average over hours, not days as misread original intent:

tt.Hour=hour(tt.Time);
>> varfun(@mean,tt,"InputVariables",'Temp',"GroupingVariables",'Hour')
ans =
  21×3 timetable
            Time            Hour    GroupCount    mean_Temp
    ____________________    ____    __________    _________
    08-Mar-2019 00:00:00      0         1            16    
    08-Mar-2019 01:00:00      1         1            15    
    08-Mar-2019 02:00:00      2         1            13    
    08-Mar-2019 03:00:00      3         1            12    
    08-Mar-2019 04:00:00      4         1            16    
    08-Mar-2019 05:00:00      5         1            12    
    08-Mar-2019 06:00:00      6         1            12    
    08-Mar-2019 07:00:00      7         1            12    
    07-Mar-2019 11:00:00     11         1            16    
    07-Mar-2019 12:00:00     12         1            14    
    07-Mar-2019 13:00:00     13         1            12    
    07-Mar-2019 14:00:00     14         1            19    
    07-Mar-2019 15:00:00     15         1            23    
    07-Mar-2019 16:00:00     16         1            14    
    07-Mar-2019 17:00:00     17         1            15    
    07-Mar-2019 18:00:00     18         1            18    
    07-Mar-2019 19:00:00     19         1            17    
    07-Mar-2019 20:00:00     20         1            13    
    07-Mar-2019 21:00:00     21         1            14    
    07-Mar-2019 22:00:00     22         1            14    
    07-Mar-2019 23:00:00     23         1            15    
>>

Best Answer

Related Solutions

MATLAB: Convert hourly temperature data into daily in the specific time interval

MATLAB: Create a single average day out of hourly data (33days)

Related Question