What is the advantage of kalman filters as an online update mechanism instead of the stochastic gradient descent?
Solved – For regression with time varying parameters, SGD or Kalman filter
gradient descentkalman filterregressionstochastic gradient descenttime series
Best Answer
Both of these things can be used in an online manner, but they do this in different ways. So they are not competitors.
The Kalman filter has two purposes. First, for a batch of data, it will yield the log-likelihood of all your observed data, assuming you are estimating a Linear-Gaussian state space model. The log-likelihood is a function of the parameters, assuming your observed data are known. Second, for online data, if you know the parameters, it will recursively compute distributions of your hidden states. When used in an online manner, it recursively calculates statistical distributions for states, assuming parameters are known.
SGD is an algorithm that takes as an input a log-likelihood function. It doesn't care what model you are using, so long as you can calculate a gradient of a loss (the loss is the negative of the log-likelihood). It is a procedure for finding your parameters that maximize (or minimize the negative of) this function. When used in an online fashion, it adjusts parameters as it sees new data. The word "stochastic" refers to the fact that it doesn't use all the data to calculate a likelihood, not to the fact that it recursively computes statistical distributions.
So both can be used in an online manner. But here the KF computes distributions of the hidden states given parameters, and SGD adjusts the parameters to become more suitable.