In simulation studies the term median bias refers to the absolute value of the deviations of an estimator from its true value (which you know in this case because it is a simulation so you choose the true value). You can see a working paper by Young (2017) who defines median bias like this in table 15, or Andrews and Armstrong (2016) who plot median bias graphs for different estimators in figure 2.
Part of the confusion (also in the literature) seems to come from the fact that there are two separate underlying problems:
- weak instruments
- many (potentially) weak instruments
The problem of having a weak instrument in a just-identified setting is very different from having many instruments where some are weak, however, the two issues get thrown together sometimes.
First of all, let's consider the relationship between the estimators that we are talking about here. Theil (1953) in "Estimation and Simultaneous Correlation in Complete Equation Systems" introduced the so-called $\kappa$-klass estimator:
$$
\widehat{\beta} = \left[ X'(I-\kappa M_Z)X \right]^{-1}\left[ X'(I-\kappa M_Z)_y) \right]
$$
with $M_Z = I-Z(Z'Z)^{-1}Z'$, for the system of equations
$$
\begin{align}
y &= X\beta + u \\
X &= Z\pi + e.
\end{align}
$$
The scalar $\kappa$ determines what estimator we have. For $\kappa = 0$ you go back to OLS, for $\kappa = 1$ you have the 2SLS estimator, and when $\kappa$ is set to the smallest root of $\det (X'X - \kappa X'M_ZX))=0$ you have the LIML estimator (see Stock and Yogo, 2005, p. 111)
Asymptotically, LIML and 2SLS have the same distribution, however, in small samples this can be very different. This is especially the case when we have many instruments and if some of the are weak. In this case, LIML performs better than 2SLS. LIML here has been shown to be median unbiased. This result comes out of a bunch of simulation studies. Usually papers stating this result refer to Rothberg (1983) "Asymptotic Properties of Some Estimators In Structural Models", Sawa (1972), or Anderson et al. (1982).
Steve Pischke provides a simulation for this result in his 2016 notes on slide 17, showing the distribution of OLS, LIML and 2SLS with 20 instruments out of which only one is actually useful. The true coefficient value is 1. You see that LIML is centered at the true value whilst 2SLS is biased towards OLS.
Now the argument seems to be the following: given that LIML can be shown to be median unbiased and that in the just-identified case (one endogenous variable, one instrument) LIML and 2SLS are equivalent, 2SLS must also be median unbiased.
However, it seems that people again are mixing up the "weak instrument" and the "many weak instruments" case because in the just-identified setting both LIML and 2SLS are going to be biased when the instrument is weak. I have not seen any result where it was demonstrated that LIML is unbiased in the just-identified case when the instrument is weak and I don't think that this is true. A similar conclusion comes out of Angrist and Pischke's (2009) response to Gary Solo on page 2 where they simulate the bias of OLS, 2SLS, and LIML when changing the strength of the instrument.
For very small first-stage coefficients of <0.1 (holding the standard error fixed), i.e. low instrument strength, just-identified 2SLS (and hence just-identified LIML) is much closer to the probability limit of the OLS estimator as compared to the true coefficient value of 1.
Once the first stage coefficient is between 0.1 and 0.2, they note that the first stage F statistic is above 10 and hence there is no weak instrument problem anymore according to the rule of thumb of F>10 by Stock and Yogo (2005). In this sense, I fail to see how LIML is supposed to be a fix for a weak instrument problem in the just-identified case. Also notice that i) LIML tends to be more dispersed and it requires a correction of its standard errors (see Bekker, 1994) and ii) if your instrument is actually weak, you will not find anything in the second stage neither with 2SLS nor LIML because the standard errors are just going to be too big.
Best Answer
If you don't have a valid instrument, then the bias can actually be worse using TSLS/IV as compared to the standard linear model. If you have a valid instrument, then your bias will be 0 on average, but your standard errors will be larger. For some information on these cases, see
Bound, John, David A. Jaeger, and Regina M. Baker. 1995 "Problems with Instrumental Variables Estimation When the Correlation Between the Instruments and the Endogeneous Explanatory Variable is Weak." Journal of the American Statistical Association. 90(430): 443--450. [jstor]