In an average (median?) conversation about statistics you will often find yourself discussing this or that method of analyzing this or that type of data. In my experience, careful study design with special thought with regards to the statistical analysis is often neglected (working in biology/ecology, this seems to be a prevailing occurrence). Statisticians often find themselves in a gridlock with insufficient (or outright wrong) collected data. To paraphrase Ronald Fisher, they are forced to do a post-mortem on the data, which often leads to weaker conclusions, if at all.
I would like to know which references you use to construct a successful study design, preferably for a wide range of methods (e.g. t-test, GLM, GAM, ordination techniques…) that helps you avoid pitfalls mentioned above.
Best Answer
I agree with the point that statistics consultants are often brought in later on a project when it's too late to remedy design flaws. It's also true that many statistics books give scant attention to study design issues.
You say you want designs "preferably for a wide range of methods (e.g. t-test, GLM, GAM, ordination techniques...". I see designs as relatively independent of statistical method: e.g., experiments (between subjects and within subjects factors) versus observational studies; longitudinal versus cross-sectional; etc. There are also a lot of issues related to measurement, domain specific theoretical knowledge, and domain specific study design principles that need to be understood in order to design a good study.
In terms of books, I'd be inclined to look at domain specific books. In psychology (where I'm from) this means books on psychometrics for measurement, a book on research methods, and a book on statistics, as well as a range of even more domain specific research method books. You might want to check out Research Methods Knowledge Base for a free online resource for the social sciences.
Published journal articles are also a good guide to what is best practice in a particular domain.