Solved – Modeling linear regression with covariate dependent error

heteroscedasticityregression

I have a model of the form $y_i = a*x_i + b + e_i$. The error terms $e_i$
are independently drawn from a distribution that depends on $x$ as well as on global parameters; however, the noise has conditional mean $0$ given $x$. The goal is to formulate models and derive good estimators for the parameters $a$ and $b$ based on sample $(x; y)$.

Can anyone suggest 1/2 models for this problem?


Edit: Since $E[e|x]=0$ implies $E[e]=0$. So the only two models I can think of are:

  1. $e \sim N(0, x^2 \sigma^2)$, and
  2. $e \sim t(0,x^2,v)$, where in the 2nd model, I am considering non-standardized $t$ distribution with scale parameter $x^2$ and $df=v$.

My question is: Are the 2 models I suggested correct and can there be any more models?

Best Answer

You could use iterative feasible generalized least squares.

Start by setting weights for each datapoint to 1, i.e. no weighting.

  1. Fit a weighted regression model for each dataset using weights.
  2. Create a single dataset combining residuals/errors and their respective x values.
  3. Fit $e_i^2 = a\cdot x_i + b$. If the noise is zero mean, $e_i^2$ is equal to the variance of the error at $x_i$.
  4. Update your weights with the squared errors prediction model
  5. Go back to 1 until convergence.