I am working on a simulation study on quantile regression. So what I did is to simulate data based on a given model, which is different from the true underlying model of the data, in other words, a mis-specified model. With the true underlying model I calculate the true quantile of the dataset(yreal). And then I calculate the conditional quantile of y given each x under the mis-specified model (yfit), and I calculate the MSE by (yfit-yreal)**2 and plot it against x. Is this the correct way of calculating MSE? And how do I calculate the bias ?
I got confused since this is a quantile regression instead of a mean regression which I usually do. To make my question clearer, I give an example:
I simulate data based on a true model, say, a linear regression model as follows, for each give quantile p:
$$
\textbf{Model 1}: y=\beta_0+\beta_1 x+\beta_2 x^2 + \sigma e \text{, where } e\sim N(0,1)
$$
In other words, I know the true $\beta_0, \beta_1, \beta_2$, and $\sigma$. So I just generate random errors and plug in the values to get a simulated dataset. Under this true model Model1, I can calculate the real quantiles of the dataset for each x over the range of x as $Q(p|x)=\beta_0+\beta_1 x+\beta_2 x^2 + \sigma \Phi^{-1}(p)$.
Then for each simulated dataset, I fit a mis-specified quantile regression model as follows using rq function in R (package:quantreg):
$$
\textbf{Model 2}: Q(p|x)=\beta_0+\beta_1 x
$$
Under Model2, with each fitted $\hat{\beta_0}$ and $\hat{\beta_1}$, I can find fitted quantile using $\hat{Q}(p|x)= \hat{\beta_0} + \hat{\beta_1} x$.
Then for each x, I can calculated the MSE to be $\left(\hat{Q}(p|x)-Q(p|x)\right)^2$, and I take the average over each simulated dataset.
Is this the correct way to calculate MSE?
How about bias and variance?
Best Answer
yes, it is correct way to calculate the mean square error of predicted conditional quantile given x. However, generally, you would need to calculate the mean square error of the estimated parameters of the quantile regression model.