BayesFactor BF01 – Correct Interpretation for Two Sample t-Test

bayesianhypothesis testingrt-test

I want to test a nullinterval with bayes. So the hypothesis I have is on the
absence of an effect. Group 1 and 2 do not differ in their mean (at least not beyond an
effect size of $\delta > 0.2$). However I am very unsure if my rural understanding of Bayesian Analasis brought me to the correct interpretation of my results.
I followed the instructions proided in the vignette for the
R package BayesFactor available here and also in this nice guide. However I am not sure if I am interpreting the resulting Bayes Factor correctly?

Here is some Simlulation Data

# Loading Packages
pacman::p_load(report, BayesFactor, tidyverse, see)
set.seed(333)
n1 <- 70
n2 <- 50
example <- data.frame(
  treatment = factor(c(
    rep("Treatment A", n1),
    rep("Treatment B", n2)
  )),
  outcome = c(
    rnorm(n = n1, mean = 7, sd = 18),
    rnorm(n = n2, mean = 13, sd = 15)
  )
)

And here the nullinterval bayes t-test

bfInterval <- ttestBF(
  formula = outcome ~ treatment,
  data = example,
  nullInterval = c(-0.2, 0.2)
)

Is my understanding now correct that the variable bfinterval no contains two bayes factors?

  • One for the null-interval $|\delta| < 0.2 $ compared to the null-point and
  • One for the complement $|\delta| > 0.2 $ of the null-interval to the null-point

And the complement (that there would be an effect > 0.2) is more likely, because BF $BF_1 = 0.18$ is farther away from 1, than the BF BF $BF_0 = 0.96$ for the null-interval? Would that be correct?

> bfInterval
Bayes factor analysis
--------------
[1] Alt., r=0.707 -0.2<d<0.2    : 0.9674333 ±0%
[2] Alt., r=0.707 !(-0.2<d<0.2) : 0.185251  ±0.01%

Against denominator:
  Null, mu1-mu2 = 0 
---
Bayes factor type: BFindepSample, JZS

Now if I want to know, if, given my data, the hypothesis that there is no effect $|\delta| < 0.2 $ is more likely than the complement (there is an effect, $|\delta| > 0.2 $ ) I can just divide the two BF since they have the same denominator correct?

> bfInterval[1] / bfInterval[2]
Bayes factor analysis
--------------
[1] Alt., r=0.707 -0.2<d<0.2 : 5.222284 ±0.01%

Against denominator:
  Alternative, r = 0.707106781186548, mu =/= 0 !(-0.2<d<0.2) 
---
Bayes factor type: BFindepSample, JZS

So this ($BF_{01} = 5.2$) would now mean, given my data, it is ~5 times more likely that there is no effect (bigger than 0.2) than that there is one, correct?

Also, beyond my insecurities in the interpretation I would like to know if the floor effect in the observed data could be problematic?

Best Answer

Both the procedure for calculating BF and its interpretation are correct.

Regarding the floor effect, it is necessary to assess how strong the asymmetry is and how large each sample size is. The t-test assumes that means difference is normally distributed in the population and that the variances of the populations to be compared are equal. Various threads in "cross-validation" discuss the assumptions of the t-test and its violation limits:

T-test for non normal when N>50?

Robust t-test for mean

As general rule, skeweness is better tolarated when sample size is larger.

Related Question