Solved – Small n: non-parametric or parametric tests

likertnonparametricsamplesample-size

I am in a situation where I have sample with $n=19$, and the the data are not normally distributed (according to the results of a Kolmogorov-Smirnov test) for all the variables. The data were collected using Likert scales in a questionnaire.
So now I am unsure whether or not I can use a parametric test or whether I need to use non-parametric tests.

I have found books stating that if you have a small $n$, you should always use non-parametric tests. However I have also found citations stating that the choice between parametric and non-parametric tests depends on the level of your data (Likert can be seen as nominal), so I should use parametric tests.

Best Answer

The data were collected using Likert scales in a questionnaire.

From the moment you decided to use Likert scales, the issue of whether or not the data were actually from a normal distribution was decided (in the negative). It's pointless to test (and answer with chance of error) a question to which you already know the answer with certainty (a large enough sample would always lead to rejection by a suitable test, but you can tell it is the case with no data at all). Your data are not from a normal distribution normal; that was already certain.

[However, it's also not a useful question to answer; a better question to answer is not 'are the data from a normal distribution?' (are they ever?) but 'how much impact does it have on my inference?', a question not answered by hypothesis tests.]

So now I am unsure whether or not I can use a parametric test

That depends on the parametric test. Parametric doesn't necessarily imply "normal"; you may be able to make some other distributional assumption that will be consistent enough with your situation that you would be content with the impact of whatever deviation from assumptions you have.

or whether I need to use non-parametric tests.

Beware - nonparametric tests also have assumptions, and in some cases may be somewhat sensitive to them.

Many nonparametric tests assume continuous data, for example, and if you don't account for heavy discreteness you may get tests with quite different properties from their nominal ones. Some assume symmetry. In addition, suitability of particular tests may depend on the precise hypothesis you're interested in - you may need some additional assumptions (or perhaps a somewhat different nonparametric procedure) to get a test of your actual hypothesis.

I have found books stating that if you have a small n, you should always use non-parametric tests.

For very small $n$, that's not necessarily useful advice, since you may have no useful significance levels available to you. At larger (but still small) $n$, in cases where the assumptions of a suitable nonparametric procedure are tenable, it sometimes makes sense to avoid making parametric assumptions to which your inferences may be sensitive (though there's sometimes the possibility of choosing different, less sensitive procedures).

However I have also found citations

Which citations? What did they actually say?

stating that the choice between parametric and non-parametric tests depends on the level of your data (Likert can be seen as nominal)

No. A single item intended as part of a Likert scale is at least ordinal. If you have constructed a Likert scale by adding a number of such questions you have already assumed it was interval - by assuming things like '5'+'2' = '4'+'3' (which must be the case if you're able to add the scores and treat every '7' as the same), every component item had to have been interval. If they're interval, their sum certainly is.

so I should use parametric tests.

I don't see how "use a parametric test" follows from that.


You say very little about what kind of hypotheses you have (what are you trying to find out?); more might be said in those circumstances.