MATLAB: Use retime() to create rows of empty data points

datetimeMATLABretimesynchronize

Hello,
I have a dataset which looks like this:
Date Time V1 V2
'29.05.2019 10:30' 1 0
'31.05.2019 08:20' 0 1
'31.05.2019 08:30' 0 1
'31.05.2019 08:40' 0 1
'31.05.2019 08:50' 0 1
'31.05.2019 09:00' 0 1
'31.05.2019 09:10' 0 1
'31.05.2019 09:20' 0 1
'31.05.2019 09:30' 0 1
Where the data jumps from time to time, I would like to insert rows so that the DateTime variable increases in 10 minute increments in each row. The associated variables V1 and V2 should be filled with zeroes, whilst the existing values are left alone.
I have tried retime and while I can get this function to produce the array of DateTimes I am looking for, I cannot do this without removing the existing data in v1 and V2. Furthermore, I cannot get an interval of ten minutes without an error.
clc;clear;
m=readtable('G:\My Drive\Results\Tawh.csv'); %read in table
m.Date=datetime(m.Date, 'Format', 'dd/MM/yyyy'); %read columns as datetime values
m.Time=datetime(datevec(m.Time),'Format','HH:mm:ss');
m.Date.Format='dd.MM.uuuu HH:mm'; %change the display format to view both date and time info.
m.Time.Format='dd.MM.uuuu HH:mm';
m.DateTime = m.Date + timeofday(m.Time); %combine the date and time in one new column
m=m(:,[4:5,7]); %remove unnecessary columns
TT = table2timetable(m, 'RowTimes', 'DateTime'); %convert to timetable
dt = minutes(10);
%TT2 = retime(TT,'regular', 'TimeStep', dt);
%creates correct time values but deletes any data from other columns
%TT.Properties
%TT.Properties.VariableContinuity = {'event','event'};
TT2=retime(TT, 'minutely', 'fillwithmissing');

Best Answer

Hi Louise,
I have tried retime and while I can get this function to produce the array of DateTimes I am looking for, I cannot do this without removing the existing data in v1 and V2.
It looks like we need to shift each value in the DateTime variable to the start of each minute (i.e. zero out the seconds part of each value in DateTime) before calling 'retime'.
Modifying the Format of TT.DateTime, we see that each value has a non-zero number of seconds. If we use 'retime' at this point, then we will lose our existing data, as you have seen.
>> TT.DateTime.Format = 'dd.MM.uuuu HH:mm:ss';
>> head(TT)
ans =
8×2 timetable
DateTime BoatCount PV
___________________ _________ __
29.05.2019 10:30:41 1 0
31.05.2019 08:20:07 0 1
31.05.2019 08:30:07 0 1
31.05.2019 08:40:07 0 1
31.05.2019 08:50:07 0 1
31.05.2019 09:00:07 0 1
31.05.2019 09:10:07 0 1
31.05.2019 09:20:07 0 1
>> temp = retime(head(TT),'regular','fillwithmissing','TimeStep',minutes(10));
>> head(temp)
ans =
8×2 timetable
DateTime BoatCount PV
___________________ _________ ___
29.05.2019 10:30:00 NaN NaN
29.05.2019 10:40:00 NaN NaN
29.05.2019 10:50:00 NaN NaN
29.05.2019 11:00:00 NaN NaN
29.05.2019 11:10:00 NaN NaN
29.05.2019 11:20:00 NaN NaN
29.05.2019 11:30:00 NaN NaN
29.05.2019 11:40:00 NaN NaN
We can fix this by zeroing the seconds part of the DateTime variable with 'dateshift'.
>> TT.DateTime = dateshift(TT.DateTime,'start','minute');
>> head(TT)
ans =
8×2 timetable
DateTime BoatCount PV
___________________ _________ __
29.05.2019 10:30:00 1 0
31.05.2019 08:20:00 0 1
31.05.2019 08:30:00 0 1
31.05.2019 08:40:00 0 1
31.05.2019 08:50:00 0 1
31.05.2019 09:00:00 0 1
31.05.2019 09:10:00 0 1
31.05.2019 09:20:00 0 1
Now, when we use 'retime', our original data won't be lost.
>> TT2 = retime(TT,'regular','fillwithmissing','TimeStep',minutes(10));
>> head(TT2)
ans =
8×2 timetable
DateTime BoatCount PV
___________________ _________ ___
29.05.2019 10:30:00 1 0
29.05.2019 10:40:00 NaN NaN
29.05.2019 10:50:00 NaN NaN
29.05.2019 11:00:00 NaN NaN
29.05.2019 11:10:00 NaN NaN
29.05.2019 11:20:00 NaN NaN
29.05.2019 11:30:00 NaN NaN
29.05.2019 11:40:00 NaN NaN
>> tail(TT2)
ans =
8×2 timetable
DateTime BoatCount PV
___________________ _________ __
18.12.2019 18:20:00 0 0
18.12.2019 18:30:00 0 0
18.12.2019 18:40:00 0 0
18.12.2019 18:50:00 0 0
18.12.2019 19:00:00 0 0
18.12.2019 19:10:00 0 0
18.12.2019 19:20:00 0 0
18.12.2019 19:30:00 0 0
The associated variables V1 and V2 should be filled with zeroes
To fill with 0's instead of NaN values for numeric variables, we can use the 'fillwithconstant' method instead of 'fillwithmissing'.
>> TT2 = retime(TT,'regular','fillwithconstant','Constant',0,'TimeStep',minutes(10));
>> head(TT2)
ans =
8×2 timetable
DateTime BoatCount PV
___________________ _________ __
29.05.2019 10:30:00 1 0
29.05.2019 10:40:00 0 0
29.05.2019 10:50:00 0 0
29.05.2019 11:00:00 0 0
29.05.2019 11:10:00 0 0
29.05.2019 11:20:00 0 0
29.05.2019 11:30:00 0 0
29.05.2019 11:40:00 0 0
>> tail(TT2)
ans =
8×2 timetable
DateTime BoatCount PV
___________________ _________ __
18.12.2019 18:20:00 0 0
18.12.2019 18:30:00 0 0
18.12.2019 18:40:00 0 0
18.12.2019 18:50:00 0 0
18.12.2019 19:00:00 0 0
18.12.2019 19:10:00 0 0
18.12.2019 19:20:00 0 0
18.12.2019 19:30:00 0 0
Best,
Seth