Which test can I use to compare a “percentage structure” (of three categories) between two groups

hypothesis testingnonparametric

Let's assume I have 3 mutually exclusive options: A, B and C and 2 groups: X and Y.

In group X I have the following percentage structure:
A – 10 (10%)
B – 85 (85%)
C – 5 (5%)

In group Y it follows:
A – 20 (20%)
B – 70 (70%)
C – 10 (10%)

I'd like to compare the 2 groups not one by one, i.e. A_X vs. A_Y, B_X vs.B_Y and so on, but rather the entire "percentage structure".

Actually I can do this for just 2 items, by calculating % of one of them with respect to the sum (like number of successes). Exemplary R code (but I don't care about software) for comparing "A to B" between X and Y

> prop.test(x = c(10, 20), n = c(10+85, 20+70))

    2-sample test for equality of proportions with continuity correction

data:  c(10, 20) out of c(10 + 85, 20 + 70)
X-squared = 3.8322, df = 1, p-value = 0.05028
alternative hypothesis: two.sided
95 percent confidence interval:
 -0.2335402588 -0.0003778699
sample estimates:
   prop 1    prop 2 
0.1052632 0.2222222

Now I can do the same for the "B to C".
I will get 2 results of tests.

But is there a statistical test (ideally if you could provide the formula/reference/book), that can compare the entire structure "A to B to C" in X vs. "A to B to C" in Y?

By the way is there anything like 2-way ANOVA for proportions/fractions? I think I could use a non-parametric test, like ANOVA-Type Statistis (ATS) or Aligned-Rank-Transform ANOVA (ART), which are both non-parametric approach to n-way ANOVA with interactions (not like just Kruskal-Wallis, which is just one-way), by treating the counts or fractions as ranked numbers. But maybe there's something dedicated? This way I could check, if:

  • there are differences between the counts (A vs. B vs. C)
  • there are differences between the groups (X vs. Y)
  • there is an interaction between them.

Best Answer

Chi-squared test of homogeneity.

First, for a test of homogeneity you need counts throughout. So suppose you have following table of counts for 200 subjects altogether; two groups, three categories:

        A    B    C   Total
X      10   85    5     100
Y      20   70   10     100
Total  30  155   15     200

If you want to know whether Groups X and Y have the same proportions for A, B and C, it seems best to start with a chi-squared test of homogeneity. Its implementation in R gives the following results:

x = c(10, 85, 5);  y = c(20, 70, 10)
TBL = rbind(x, y);  TBL
  [,1] [,2] [,3]
x   10   85    5
y   20   70   10
chisq.test(TBL)

        Pearson's Chi-squared test

data:  TBL
X-squared = 6.4516, df = 2, p-value = 0.03972

The P-value of about 4% indicates that there is (barely) a significant difference at the 5% level of significance.

Expected counts reflect what one would expect if groups X and Y are homogeneous. It is a good idea to check to see that these counts are large enough for the 'chi-squared' statistic to have a chi-squared distribution. [The program will show a warning message if they are too small.] One hopes to see all expected counts greater than 5, which is the case here.

chisq.test(TBL)$exp
  [,1] [,2] [,3]
x   15 77.5  7.5
y   15 77.5  7.5

If you get a significant P-value, it can be useful to look at the Pearson residuals. The sum of squares of the Pearson residuals is the chi-squared statistic $6.4516,$ shown in the results above.

chisq.test(TBL)$resi
       [,1]       [,2]       [,3]
x -1.290994  0.8519428 -0.9128709
y  1.290994 -0.8519428  0.9128709

The largest of them may point the way to the most interesting differences among sample proportions for the two groups. These might be explored in ad hoc tests. Usually, one looks for residuals with absolute values of about 2 or above, but there are no such residuals here. However, in order to explore the idea, let's compare counts in A and B for the two groups. [It seems this may be essentially what you did with prop.test in your Question.] This particular result is not significant even at the 5% level.

chisq.test(TBL[,c(1,2)])$p.val
[1] 0.05027749

Moreover, in order to avoid 'false discovery', from repeated analyses of the same data, you should use some method (such as Bonferroni's) to do multiple ad hoc tests at even lower significance levels.

Notes: (1) If you had a lot more than 200 subjects overall, with about the same sample proportions as in your TBL, you might see some additional significant results.

For example, if you used fake data in TBL.FAKE, with an imaginary 1000 subjects, then both the overall chi-squared test and the ad hoc test on just counts A and B would be highly significant with P-values near $0.$ This illustration with fake data emphasizes that you need to analyze counts, not proportions.

Bar charts of data in TBL and TBL.FAKE would look exactly the same, except possibly for counts shown on a vertical axis or total counts shown in a footnote.

TBL.FAKE = 5*TBL
TBL.FAKE
  [,1] [,2] [,3]
x   50  425   25
y  100  350   50

sum(TBL.FAKE)
[1] 1000

chisq.test(TBL.FAKE)$p.val
[1] 9.89123e-08
chisq.test(TBL.FAKE[ ,c(1,2)])$p.val
[1] 2.198613e-06

(2) You may want to look in a standard elementary or intermediate-level applied statistics text if you are not familiar with such chi-squared tests.

Also, there are many examples of chi-squared tests on this site. You might use the search line at the top of the page to search for 'chi-squared test, homogeneity'. You might start by looking at the pages marked as 'Relevant' in the margin of this page.