Solved – Troubling p value in ANOVA analysis

anovap-value

I performed an ANOVA analysis using the population genetic software Arlequin.
I obtained, from some data set, a weird result:

----------------------------------------------------------------------
 Source of                  Sum of      Variance         Percentage
 variation      d.f.        squares     components       of variation
----------------------------------------------------------------------
 Among
 groups           1          2.253        0.04725 Va            21.07

 Among
 populations
 within
 groups           4          1.515        0.01703 Vb             7.60

 Within
 populations     73         11.675        0.15993 Vc            71.33
----------------------------------------------------------------------
 Total           78         15.443        0.22421
----------------------------------------------------------------------
 Fixation Indices
  FSC :      0.09623
  FST :      0.28667
  FCT :      0.21072
----------------------------------------------------------------------

 Significance tests (1023 permutations)
------------------

 Va and FCT : P(rand. value > obs. value)  =  0.00000
              P(rand. value = obs. value)  =  0.10753
              P(rand. value >= obs. value) =  0.10753+-0.01354

How is it possible that the first $p$ value equals zero when the second is $0.10753$?

Best Answer

Arlequin performs a permutation test to generate the p-values for the test. This can be seen from the output that states that the p-values are based on 1023 permutations.

The permutation test could go roughly as follows. First you calculate the FCT for the original data and note the result. Then you randomize (permute) the data, in this case 1023 times, and calculate FCT for each of these data sets, and get 1023 new FCT values. Then you compare the FCT values from the permuted samples to the original, and note how many of permuted FCT values are equal to, or equal to or larger to the FCT calculated from the original data.

In this case, none of the permuted FCT values were larger than the original one (P(rand. value > obs. value) = 0.0000), but about every tenth were equal to the original FCT value (P(rand. value = obs. value) = 0.10753). As the p-value from a randomization test is often interpreted to be the frequency of permuted values that are larger than or equal to the original value, your p-value would appear to be 0.10753 for that particular test.

As to why that particular test behaves in such a way, it's hard to tell for sure. There probably are some features in the data that lead to this behavior.

Related Question