RNNs are remarkably good for capturing the time-dependence of sequential data. However, what happens when the sequence elements aren't equally spaced in time?
E.g., the first input to the LSTM cell happens on Monday, then no data from Tuesday to Thursday, and then finally new inputs for each of Friday, Saturday, Sunday. One possibility would be to have some kind of NULL vector being fed for Tuesday through Thursday, but that seems to be a silly solution, both because the NULL entries will contaminate the data and because it's a waste of resources.
Any ideas? How do RNNs handle such cases? If there are methods other than RNNs, I welcome those suggestions as well.
Best Answer
I just wrote a blog post on that topic!
In short, I write about different methods for dealing with the problem of sparse / irregular sequential data.
Here is a short outline of methods to try:
Lomb-Scargle Periodogram
This is a way of computing spectrograms on non-equidistant timestep series.
Data modeling with Interpolation networks
You really don't want to interpolate naively between timesteps, but training a network to interpolate for you might help!
Neural Ordinary Differential Equation models
Neural networks that can work with continuous time can naturally work on irregular time series.
Add timing dt to the input as an additional feature (or positional encoding in Tensorflow)
Methods for dealing with missing values
This is only viable if you have vast amounts of data
Hope this helps point you to the right direction :)