I have got a total sample size of 12 soil samples which were split into 4 groups of $n=3$.
A number of different variables were measured like content of organic carbon. These data are logistically difficult and expensive to collect, so collecting more data is not possible. Now I want to compare the means of the 4 groups ($n=3$) by testing. But there is the problem that the normal distribution (Shapiro-Wilk) and homoscedasticity (Brown-Forsythe/Levene) is not given in each sample. So which test should I prefer? What do you think? Is it even useful to use test statistics in this case? Are there any alternatives beneath descriptive statistics?
OK, here are some details. The soil samples are from a long-term experiment site with four different treatments (plough, no tillage and so on). From each of these four treatments I have 3 soil samples. The different variables are measured in metric data.
For example – content of organic carbon [g/kg]:
(Group 1: 12.1 // 13.2 // 13.5),
(Group 2: 13.1 // 13.9 // 13.5),
(Group 3: 8.9 // 10.2 // 11.9).
Now I wanted to test if there are any significant differences among the medians of these groups. In addition, concentrations of different substances in soils are generally not normal distributed.
Best Answer
Please refer to questions on testing for assumptions of your model. They are usually a pointless endeavour and particularly so in this case.
Simple parametric tests, such as t-test and ANOVA, were designed for small samples sizes. So, there's not necessarily any good reason to be considering other tests. But one would really need much more information to offer better advice.
EDIT: You have 9 data points in total. Just talk about the raw data and ignore statistical summaries. The issue with the fundamental assumption of representativeness, for any discussion of the data, is going to be your biggest problem. Non-parametric tests require bigger Ns than you have and parametric ones require more information. From the values you have I'd just say that groups 1 and 2 are easily samples from the same soil but for group 3 it might be a little less and argue for collecting more data.