My task is to evaluate the effectiveness of an intervention. In my study, there are one control group and one intervention group. There are two time points, baseline and endpoint.
And I use the score of some questionnaire as outcome measures.
My strategy to show/test the effectiveness is to compare the change between baseline and endpoint between these two groups by independent t test for continuous measures.
I calculated both the point difference $(\text{endpoint}-\text{baseline})$ and relative difference $((\text{endpoint}-\text{baseline})/\text{baseline})$. After running t test in SPSS, I got two different result that the $p$-value of the $t$-test on point difference was higher than $0.05$ and $p$-value of the $t$-test on relative difference is lower than $0.05$.
These results were contradicting because I set the alpha as $0.05$.
I want to ask: which difference should I use?
Best Answer
Percent change is an asymmetric measure that is not valid to compute statistics on. You would need to analyze the log ratio if you think relative changes are more appropriate than absolute changes. But the central question is what are the properties of the measurement and what is the correct model to use. A Bland-Altman plot can help you answer the question, i.e., to choose the appropriate transformation. Plot y-x vs. (x+y)/2 or log(y/x) vs. geometric mean of x and y. See if one of these is flat in central tendency with flat variability.
But note that pre-post designs are seldom valid for inference unless you had the ability to freeze all external conditions so that nothing else changed between baseline and the post-baseline measurement.