Intro:
I have recently made a simulation program which simulates patients with Type 1 diabetes. In that context I am creating artifical patients. Let's denote three of these as $\text{p}_1$, $\text{p}_2$, and $\text{p}_3$. Assume that $\text{p}_1$, $\text{p}_2$ and $\text{p}_3$ are treated with treatment $A$. Treatment $A$ treats $\text{p}_1$, $\text{p}_2$ and $\text{p}_3$'s diabetes to a certain degree. We evaluate the treatment with some measures conducted in a virtual trial; let's denote one of these measures $\text{M}1_A$. $\text{M}1_A$ has an average value and a standard deviation for Treatment A for the three patients.
Now let's introduce treatment B for the same patient group, $\text{p}_1$, $\text{p}_2$ and $\text{p}_3$. Again we put the treatment through a trial test and receive a list of measures, this time denoted $\text{M}1_B$. I am interested in testing whether the difference in $\text{M}1_A$ and $\text{M}1_B$ is significant.
I know:
$N=3, \text{std(M}1_A), \text{std(M}1_B), \text{mean(M}1_A), \text{mean(M}1_B)$
Question:
I want to compare the two measures tested on the same group of 'people' and use a statistical tool to figure out if the difference in the two measures are significant.
I am considering using Welch's t-test – would this be correct?
Best Answer
If they are the same three patients, the measurements are paired, in which case a Welch t-test is not ideal. A paired test of some kind, perhaps a paired t-test, a permutation test (that deals with the pairing), or a Wilcoxon signed rank test might be suitable, perhaps.
(You might also need to worry about order effects in your design.)