MATLAB: How to create a new matrix adding vectors of different size based on time

matrix manipulationtime series

Hi,
Wondering if I could get some ideas, or guide me to a similar example, on how I can organize 300 times series of data, each time series has data in column 1 (numerical, double) and date/time (01-Jan-2000 00:00:00) in column 2. These time series can have from ~10,000 to ~115,000 data points, and can have both different start and end date/time and time steps between data points. Ideally I would like to end with a matrix that can have in column 1 the date/time, and then consecutive 300 columns with the data organized respect to date/time (column 1).
I believe my first step should be to use the function datetime to create a vector (column 1) with NaN's having the star and end date/time from the longest time series of these 300 time series, and an interval of time that represent the smallest time step from the time series. Then, add it as column 1 to a large matrix generated with just NaN’s that has the # of rows from column 1 and a total of 301 columns (date/time column plus 300 data columns). Thus a “background” matrix is generated. This is something I can do.
But my specific question is how I could run a loop that extract the index, based on date/time, for each of the 300 time series, and then insert the data of each time series into the “background” matrix, column by column.
This is a simple example of how my data looks like (date/time represented as serial date number from matlab):
 730486.000  0.500    730486.014  0.800    730486.000  0.500
 730486.007  NaN    730486.028  4.000    730486.021  NaN
 730486.014  0.700    730486.042  2.100    730486.042  0.700
 730486.021  3.000    730486.056  4.100    730486.063  3.000
 730486.028  2.000                  730486.083  2.000
 730486.035  4.000                  730486.104  4.000
 730486.042  1.000                  730486.125  1.000
 730486.049  0.100                  730486.140  0.100
 730486.056  10.000                  730486.167  10.000 
 730486.063  -1.000
And this is what I would like to generate:
 730486.000  0.500  NaN  0.500
 730486.007  NaN  NaN  NaN
 730486.014  0.700  0.800  NaN
 730486.021  3.000  NaN  NaN
 730486.028  2.000  4.000  NaN
 730486.035  4.000  NaN  NaN
 730486.042  1.000  2.100  0.700
 730486.049  0.100  NaN  NaN
 730486.056  10.000  4.100  NaN
 730486.063  -1.000  NaN  3.000
 730486.069  NaN  NaN  NaN
 730486.076  NaN  NaN  NaN
 730486.083  NaN  NaN  2.000
 730486.090  NaN  NaN  NaN
 730486.097  NaN  NaN  NaN
 730486.104  NaN  NaN  4.000
 730486.111  NaN  NaN  NaN
 730486.118  NaN  NaN  NaN
 730486.125  NaN  NaN  1.000
 730486.132  NaN  NaN  NaN
 730486.139  NaN  NaN  NaN
 730486.146  NaN  NaN  0.100
 730486.153  NaN  NaN  NaN
 730486.160  NaN  NaN  NaN
 730486.167  NaN  NaN  10.000

Best Answer

Hey Robert!
First, a conceptual question: Can you make sure you have the same sampling interval & phase across all your signals (e.g. are the timestamps always like in your example?). If not, things will get really complicated (you have to resample which might increase your data amount by A LOT). If it is, then I would do two steps:
  1. Create a nTimeStampsTotal x nSignals data-matrix, where nTimeStampsTotal is the number of total timestamps (e.g. samples) from the first to the last timestamp among all signals (again: assuming constant sampling rate & phase) and nSignals is the number of signals.
  2. For each signal's timestamp find the according sample in the background matrix and assign them to yourmatrix.
The following code snippet exemplifies this.
data = NaN(nTimeStampsTotal,nSignals); % Data matrix.
for i = 1:nSignals
   isSample = ismember(signalTimestamps{i},bgTimestamps); % Signal timestamp cell & background timestamp matrix.
   data(isSample,i) = signalData{i};                      % Signal data cell.
end
Be aware, that the ismember call assumes that ALL your timestamps in the signal are present in the background matrix!
Hope that helps, Greetings, David
Related Question