Let's say that we are measuring the tracking position error of two controllers and we want to know if both controllers are similar (we already perform a "differences" test but we are requested to perform an equivalence test).
For example, if the mean $\mu$ (in meters) of one controller (12 samples) is 0.00344 (standard deviation $\sigma=0.0006481424$), the other (12 samples) is 0.00331 ($\sigma=0.000498$).
If $\varepsilon$ is the magnitude of region similarity. How do we define this $\varepsilon$?
I could consider $\varepsilon$ as $N \cdot \sigma$ as the region of similarity so, for example if $N=3$ then $\varepsilon=3\cdot(0.00064)=0.0019$ (using the data of the first controller).
Or, I could use something like $\varepsilon=\mu \cdot 0.05$, this takes 5% of the mean (of the first controller) which gives $\varepsilon = 0.0172$.
The question is, should we use the data for controller 1 or 2—as in the previous options—to define this parameter; or should do we consider other kind of difference measure between them in order to determine the epsilon parameter?.
Finally, why not use the epsilon as a 0.9 which may indicate that the 90% of the 'data (or the area of the distribution)' should be similiar?
I appreciate any advice in order to determine the epsilon.
Please see e.g. the function here:
https://github.com/cran/equivalence/blob/master/R/rtost.R
http://www.inside-r.org/packages/cran/equivalence/docs/rtost
Thanks
Best Answer
Since no one answered, I will try to show what I understood and how it should be choosen the region of similarity.
The null hypothesis is $H_0 : |\mu_1 - \mu_2| > \varepsilon$ (where $\mu_1$ and $\mu_2$ are means for controller 1 and 2). Thus $\varepsilon$ is the minimum value by which we will define how similar is the response (tracking error) of the controller.
Thus, I think that due to the equipment and the resolution of the mechanical device, $\varepsilon$=1mm is a good choice. But this value is defined subjectively, by experience or taken into account any advice on the particular field of interest.