I would like to use a binary logistic regression model in the context of streaming data (multidimensional time series) in order to predict the value of the dependent variable of the data (i.e. row) that just arrived, given the past observations. As far as I know, logistic regression is traditionally used for postmortem analysis, where each dependent variable has already been set (either by inspection, or by the nature of the study).
What happens in the case of time series though, where we want to make prediction (on the fly) about the dependent variable in terms of historical data (for example in a time window of the last $t$ seconds) and, of course, the previous estimates of the dependent variable?
And if you see the above system over time, how it should be constructed in order for the regression to work? Do we have to train it first by labeling, let's say, the first 50 rows of our data (i.e. setting the dependent variable to 0 or 1) and then use the current estimate of vector ${\beta}$ to estimate the new probability of the dependent variable being 0 or 1 for the data that just arrived (i.e. the new row that was just added to the system)?
To make my problem more clear, I am trying to build a system that parses a dataset row by row and tries to make prediction of a binary outcome (dependent variable) , given the knowledge (observation or estimation) of all the previous dependent or explanatory variables that have arrived in a fixed time window. My system is in Rerl and uses R for the inference.
Best Answer
There are two methods to consider:
Only use the last $\mathrm{N}$ input samples. Assuming your input signal is of dimension $\mathrm{D}$, then you have $\mathrm{N} \times \mathrm{D}$ samples per ground truth label. This way you can train using any classifier you like, including logistic regression. This way, each output is considered independent from all other outputs.
Use the last $\mathrm{N}$ input samples and the last $\mathrm{N}$ outputs you have generated. The problem is then similar to Viterbi decoding. You could generate a non-binary score based on the input samples and combine the score of multiple samples using a Viterbi decoder. This is better than method 1. if you know something about the temporal relation between the outputs.