Hypothesis Testing – How to Test Equivalence for Consistency Validation

equivalencehypothesis testingMATLABp-value

I am testing the output of a low-power cell battery and then I measured the output after a while. I did multiple runs and the output was slightly different every time. I would like to check whether my collected data are aligned with the population and show that my output measurements are reliable and consistent. The standard is to show a P-value of P < 0.05.

My approach (MATLAB):

mu = 2.366;     % Population mean 
sample=[2.180213,2.178298   ,2.310851   ,2.114255   ,3.012553   ,2.69234    ,2.079787];
n = numel(sample);
[h,ptest] = ttest(sample,mu,0.05,'right')

(NOTE: all values are in micro *10^-6)

I am getting a value of p = 0.49 & h = 0. h = 0 indicates that ttest does not reject the null hypothesis at the 5% significance level.

Is there something I am doing wrong or are those the right p & h values for my output? can I reduce the sensitivity for my test to get better results?

Best Answer

I think that you are carrying out the wrong test for your real hypothesis. From your comments, you want to show that your numbers are "close together" and not "far apart". This is a statement about the standard deviation, not the mean. Your test right now can only tell you about the mean and not about the standard deviation. Let me give you two examples:

2.369 2.384 2.373 2.361 2.362 2.356 2.356 (mean = 2.366, standard deviation = 0.01)

4.478 0.358 3.375 3.559 1.632 2.430 0.730 (mean = 2.366, standard deviation = 1.54)

I'm guessing that you would find the first sample "acceptably close together" and the second "too far apart". If this is right, you'll need to do a chi-square test for standard deviation.

Your next steps are:

  1. Decide how large a standard deviation is acceptable to you (0.01? 0.1? 0.5?). Note that you CANNOT get this from your sample data. It needs to be a number that you choose for reasons other than the data you collected.
  2. Carry out a one-sided chi-square test with H0: standard deviation >= X vs HA: standard deviation < X, where X is the largest standard deviation you would be happy with.
  3. If you reject the null hypothesis with p < 0.05, you can say your battery power outputs are sufficiently consistent for you.

Unfortunately I am not familiar with MATLAB and so can't implement step 2, but hopefully this gives you a better idea on what to search for.

Related Question