I'm not sure, but I think I've read somewhere that because the Classical Linear Regression model assumes to have a random sample, when researchers they might not be in presence of a sample with that property, they try to use some randomization technique to make sure the usual theory maybe applied.
In Hayashi's Econometrics, chapter 2, he develops the OLS and studies its large-sample properties in a generalization of iid sample. He assumes that the sample is ergodic, and stationary, and that the regressors multiplied by the error terms (the same that define the orthogonality conditions) also follow a martingale difference sequence.
My question is, given a sample, is there a way to know if it satisfies ergodicity, stationarity, and martingale diff. seq.? Also, even if the sample does not satisfy it, is there a way to make sure that we are able to obtain such a sample, when randomization techniques are not possible to apply?
Any help would be appreciated.
P.S.: This question is also posted in Economics Beta, but there I asked for a more applied, experience-based answer.
Best Answer
I will demonstrate using an AR(1) model. Consider a variable, $\left(y_{t}\right)_{t=0,1,\ldots} $. Then consider the AR(1) model: $y_{t}=\rho y_{t-1}+\varepsilon_{t},\quad\varepsilon_{t}\thicksim N\left(0,\,\sigma^{2}\right) $.
Solving the AR(1) model we get:$y_{t}=\rho^{t}y_{0}+\sum_{i=0}^{t-1}\rho^{i}\varepsilon_{t-i} $
where $y_{0} $ is the initial value of the process. Finding the variance we get: $\sigma^{2}\frac{1-\rho^{2t}}{1-\rho} $. Clearly this process is not stationary nor weakly mixing (hence its not ergodic since weakly mixing imlies ergodicity) as both the conditional mean and the conditional variance depend upon time. We can therefore write:$y_{t}=\rho^{t}y_{0}+\sum_{i=0}^{t-1}\rho^{i}\varepsilon_{t-i}\stackrel{D}{=}N\left(\rho^{t}y_{0},\,\sigma^{2}\frac{1-\rho^{2t}}{1-\rho^{2}}\right) $
If however the condition $\left|\rho\right|<1 $ holds then the series is stationary and weakly mixing (hence ergodic since weakly mixing implies ergodicity) and we can write it as:$y_{t}=\sum_{i=0}^{t-1}\rho^{i}\varepsilon_{t-i}\stackrel{D}{=}N\left(0,\,\sigma^{2}\frac{1}{1-\rho^{2}}\right) $
since when $\left|\rho\right|<1 $ holds then $\rho^{t}\rightarrow0 $ exponentially fast and we have that $\rho^{t}y_{0}\rightarrow0 $ and $\sigma^{2}\frac{1-\rho^{2t}}{1-\rho^{2}}\rightarrow\sigma^{2}\frac{1}{1-\rho^{2}} $ exponentially fast. To summarize, when $\left|\rho\right|<1$ and $\varepsilon_{t}$ is normally distributed then the condition $\left|\rho\right|<1$ is enough to ensure ergodicity of both the mean and the variance. Now we know that $\left|\rho\right|<1 $ is enough to ensure stationarity and ergodicity of the AR(1) process and in order to test this we could run a unit root test on the series and see if the series had a unit root or not. Other sources of non-stationarity could be a deterministic trend, changing variance and structural breaks, all of which we could model. A unit root however needs a transformation of the data or we could use cointegration analysis if we had more than one series.
To illustrate I will simulate the AR(1) model: $y_{t}=y_{t-1}+\varepsilon_{t} $:
We see that the autocorrelation function decays very slowly and from the time series graph the series does not look very stationary. Therefore we can say that clearly the assumption about stationarity and weakly mixing (hence ergodicity) does not hold when $\left|\rho\right|=1 $. For information about the autocorrelation function see here.
For the second illustration I will simulate the AR(2), $y_{t}=0.3y_{t-1}+0.4y_{t-1}+\varepsilon_{t} $ model but I will try to model it with an AR(1) model to see how the residuals look like.
We see that the time series graph looks stationary and the ACF goes exponentially towards zero indicating stationarity and weakly mixing. By fitting an AR(1) model to the data we get the following model: $y_{t}=0.54y_{t-1}+\varepsilon_{t} $. If we now look at the residual ACF we see that they are not white noise (a martingale diff. seq. is white noise) and the portmanteu test rejects the null of no autocorrelation.
If we then fit an AR(2) to the simulated data we get an estimated model of: $y_{t}=0.37y_{t-1}+0.32y_{t-1}+\varepsilon_{t} $. Now the residuals do look like white noise and we cannot reject the null of no autocorrelation.
If this did not answer your question then let me know.