Bayesian Inference – Can the Bayes Factor Be Negative?
bayesianinference
This is what I saw in a source I am referring to:
Since both the numerator and the denominator are probabilities (so they can only take any value between 0 and 1), how can the result of division be negative?
Best Answer
Not sure what your source is, but whoever it is seems to have botched Harold Jeffreys' cutoffs. The items in the table do match his recommendations, but with two problems. The first is that the cutoffs are intended to be for $\log \text{BF}$ rather than for the Bayes factor itself (EDIT: slightly misspoke, the unit is "decihartleys" which corresponds to $10 \log_{10}(\text{BF})$). The second is that, probably because the person who put the table together didn't realize the scale was suppose to be $\log$, they "corrected" the first row to be $< 1$ instead of $< 0$ because they assumed $< 0$ must be a typo (since Bayes factors can't be negative).
We have that (ref)
$$
\frac{\text{RSS}}{\sigma²} \sim \chi²_{n - p}
$$
Under the null hypothesis that all parameters are zero, we also have that
$$
\frac{\text{ESS}}{\sigma²} \stackrel{\small \text{H}_0}{\sim} \chi²_p
$$
RSS and ESS are independent.
You are proposing to consider the ratio of $\frac{\text{ESS}}{\sigma²}$ and of $\frac{\text{RSS}}{\sigma²}$,
$$
\cfrac{\frac{\text{ESS}}{\sigma²}}{\frac{\text{RSS}}{\sigma²}} = \frac{\text{ESS}}{\text{RSS}} = \frac{p}{n-p} \times \cfrac{\frac{\text{ESS}}{n-p}}{\frac{\text{RSS}}{p}}
$$
which, under the null, is distributed as a constant multiple of an F distribution (as noted by @Whuber in his comment to the previous version of my answer).
In summary, what you propose is not really different from what is done.
From a technical point of view, here is the argument:
For densities (but the argument is analogous in the discrete case), we write
$$ \pi \left( \theta |y\right) =\frac{f\left( y|\theta \right) \pi \left(\theta \right) }{f(y)}
$$
The norming constant can be obtained as, by writing a marginal density as a joint density and then writing the joint as conditional times marginal, with the other parameter integrated out,
\begin{align*}
f(y)&=\int f\left( y,\theta \right) d\theta\\
&=\int f\left( y|\theta \right) \pi \left(\theta \right)d\theta
\end{align*}
It ensures integration to 1 because
\begin{align*}
\int \pi \left( \theta |y\right) d\theta&=\int\frac{f\left( y|\theta \right) \pi \left(\theta \right) }{\int f\left( y|\theta \right) \pi \left(\theta \right)d\theta}d\theta\\ &=\frac{\int f\left( y|\theta \right) \pi \left(\theta \right) d\theta}{\int f\left( y|\theta \right) \pi \left(\theta \right)d\theta}\\
&=1,
\end{align*}
where we can "take out" the integral in the denominator because $\theta$ had already been integrated out there.
Best Answer
Not sure what your source is, but whoever it is seems to have botched Harold Jeffreys' cutoffs. The items in the table do match his recommendations, but with two problems. The first is that the cutoffs are intended to be for $\log \text{BF}$ rather than for the Bayes factor itself (EDIT: slightly misspoke, the unit is "decihartleys" which corresponds to $10 \log_{10}(\text{BF})$). The second is that, probably because the person who put the table together didn't realize the scale was suppose to be $\log$, they "corrected" the first row to be $< 1$ instead of $< 0$ because they assumed $< 0$ must be a typo (since Bayes factors can't be negative).