Let's say that I have a linear system (e.g. an electrical circuit) and I am trying to estimate the values of parameters in the system (e.g. the resistance, inductances, and capacitances). I do this by measuring the response of the system to some input signal (e.g. a step function). If the noise in the measurement were uncorrelated and Gaussian, then one obvious approach to parameter estimation would be to use the maximum-likelihood estimator, which would reduce to a (non-linear) least square fit. Simple enough.
However, if the measurement is very sensitive, so that the dominant source of noise comes from compoments internal to the system (e.g. Johnson noise in a resistor), the noise at the output will be correlated. This means that maximum likelihood estimator for the parameters will not reduce to a simple least squares procedure.
I'm sure that there is a set of statistical practices and literature devoted to parameter estimation for systems like this, but I'm not sure where to start looking. Can anyone point me in the right direction?
Update I just want to clarify that although the system is linear in the sense that a linear superposition of two signals is also a valid signal, the output of the system as a function of time is certainly not linear. In general it would be some sum of exponentials, where the coefficients in front of each exponential are function of the various time constants.
Best Answer
So if you have a time series model for the measurements
X(t) = f(previous Xs and ts) + N(t) where N(t) = noise at time t.
then you can write N(t) =e$_t$ + a$_1$ e$_t$$_-$$_1$ +a$_2$ e$_t$$_-$$_2$ +...+ a$_k$ e$_t$$_-$$_k$
where the e$_t$s are iid random noise (i.e. white noise) Then N(t) is correlated noise with autocorrelation determined by the a$_i$s (k could be just 1 or it could be 2 or higher).
Given a model like this you can write down the likelihood and find the maximum. Being that you say f is linear this is solved via conditional least squares.