I have data from an EEG (electrophysiological data) experiment where we have more than 10 electrodes recording brain activity (dependent variable) simultaneously. The design is a within subject design with four conditions (independent variables) presented to each subject. I collect EEG amplitudes as dependent variables. For each condition, subjects saw 40 trials.
We are interested in finding which electrodes show statistically significant differences between some of the pairwise comparisons between these four conditions (i.e., A-B, A-C, A-D or A-B, B-C, C-D). There are at least three comparisons like that we are interested in.
I would like to apply permutation or another nanparametric test in order to achieve correction for multiple comparisons for both i) condition comparisons as well as ii) across electrodes. Because I take each electrode as a separate measure and I believe I should correct for family-wise error. One thing to note is that electrode activities may not be fully independent from each other due to the nature of the voltage distribution across scalp (see below).
Many permutation examples use an independent variable with two conditions, and they permute these conditions for each subject for each electrode, and take the maximum t value across electrodes in each permutation, repeating this process 1000 times, and get a tmax distribution, then finding the 95th percentile value as a threshold. This way they control for false discovery rate across electrodes. Some other methods use cluster based method, where they use the spatial proximity of the electrodes and find the minimum electrode cluster size as a threshold. This method considers dependence between electrodes, but overlooks the small but significant clusters. But, all these methods assume simple two condition cases, swapping condition values for permutation test.
My question is: Which nonparametric method would be wise to apply in my situation where I have four within subject conditions as well as many electrodes that these measures are collected?
Thanks
Best Answer
My PhD thesis was on precisely the topic of how to best test for significant differences in EEG and I faced the same questions.
I found the optimal method is to use a mass-univariate test for each electrode and time/frequency point independently. This may be a t-test, or ANOVA (as in your case a repeated measures ANOVA), or even simply the mean differences, as long as you can justify that the test is valid for your hypothesis.
Then, precisely because there is a high correlation between neighbouring electrodes, and neighbouring time points, the 'trick' is to use the neighbourhood to enhance your original uni-variate measure (e.g. t-values), using threshold-free cluster-enhancement. Essentially this looks at the intensity of the test statistic and scales this value according to the corresponding strength of neighbouring values.
See my NeuroImage paper for details. There I show how the method is logically superior to existing methods, but also use simulations to show precisely its sensitivity and specificity against the methods you already mentioned as well as Statistical Parametric Mapping and Global Field Power.
I've created a user-friendly matlab toolbox to quickly and easily use the method. All you need is your data and the spatial locations of your electrodes and you're ready to go.
You can find the toolbox on Github here.
Feel free to contact me with any questions regarding the method.