Solved – Aside from Durbin-Watson, what hypothesis tests can produce inconclusive results

decision-theoryhypothesis testingstatistical significance

The Durbin-Watson test statistic can lie in an inconclusive region, where it is not possible either to reject or fail to reject the null hypothesis (in this case, of zero autocorrelation).

What other statistical tests can produce "inconclusive" results?

Is there a general explanation (hand-waving is fine) for why this set of tests are unable to make a binary "reject"/"fail to reject" decision?

It would be a bonus if someone could mention the decision-theoretic implications as part of their answer to the latter query — does the presence of an additional category of (in)conclusion mean that we need to consider the costs of Type I and Type II errors in a more sophisticated way?

Best Answer

The Wikipedia article explains that the distribution of the test statistic under the null hypothesis depends on the design matrix—the particular configuration of predictor values used in the regression. Durbin & Watson calculated lower bounds for the test statistic under which the test for positive autocorrelation must reject, at given significance levels, for any design matrix, & upper bounds over which the test must fail to reject for any design matrix. The "inconclusive region" is merely the region where you'd have to calculate exact critical values, taking your design matrix into account, to get a definite answer.

An analogous situation would be having to perform a one-sample one-tailed t-test when you know just the t-statistic, & not the sample size: 1.645 & 6.31 (corresponding to infinite degrees of freedom & only one) would be the bounds for a test of size 0.05.

As far as decision theory goes—you've a new source of uncertainty to take into account besides sampling variation, but I don't see why it shouldn't be applied in the same fashion as with composite null hypotheses. You're in the same situation as someone with an unknown nuisance parameter, regardless of how you got there; so if you need to make a reject/retain decision while controlling Type I error over all possibilities, reject conservatively (i.e. when the Durbin–Watson statistic's under the lower bound, or the t-statistic's over 6.31).

† Or perhaps you've lost your tables; but can remember some critical values for a standard Gaussian, & the formula for the Cauchy quantile function.

Related Question