I'm a grad student in psychology, and as I pursue more and more independent studies in statistics, I am increasingly amazed by the inadequacy of my formal training. Both personal and second hand experience suggests that the paucity of statistical rigor in undergraduate and graduate training is rather ubiquitous within psychology. As such, I thought it would be useful for independent learners like myself to create a list of "Statistical Sins", tabulating statistical practices taught to grad students as standard practice that are in fact either superseded by superior (more powerful, or flexible, or robust, etc.) modern methods or shown to be frankly invalid. Anticipating that other fields might also experience a similar state of affairs, I propose a community wiki where we can collect a list of statistical sins across disciplines. Please, submit one "sin" per answer.
Solved – What are common statistical sins
fallacy
Related Solutions
The universe and mankind
The fact that we observe the unlikely event of a universe, solar system and planet that is able generate intelligent life, is a type of survival bias.
(and as Mehmet mentions in the comments, this could be seen as a cherry picking fallacy)
We see the unlikely event because without it we wouldn't have lived to see the absence of the event.
In relationship with the existence of life this phenomenon relates to the anthropic principle. This has different forms. From Barrow and Tipler's book:
Weak anthropic principle (WAP): The observed values of all physical and cosmological quantities are not equally probable but they take on values restricted by the requirement that there exist sites where carbon-based life can evolve and by the requirements that the Universe be old enough for it to have already done so.
The world and universe are not very probable, but it is the way it is because there is a selection effect.
Strong anthropic principle (SAP): The Universe must have those properties which allow life to develop within it at some stage in its history.
In this case, the properties of the universe are not regarded as probability but as some sort of intelligent design (this is also more like a philosophical argument, involving teleological ideas, and may not be so much of a fallacy). But it can be even adapted further. With interpretations of quantum dynamics, it is not just that the special unlikely conditions for the universe are necessary for humans/observes, but also the other way around observers are necessary for the universe to exist.
The zero probability
Suppose I threw randomly a dart in a circle and it hit exactly at the point (3, 4), if one says for me: "There infinity many points you could hit, so, if you threw it randomly, the probability (using the limit definition) of you hitting that point is 0, therefore you didn't throw it randomly and must have something external being that rationally made this dart hit (3,4)".
This argument is slightly different from the survival bias. The argument is that events with zero probability are not supposed to happen.
This is misrepresenting 'probability' (which we could see as an equivocation fallacy).
While the result/outcome 'the dart hits (3,4)' is part of the sample space, the set of all possible outcomes (often denoted as $\Omega$), it is not in the event space. The result/outcome 'the dart hits (3,4)' is not an event in event space to which we can formally assign a probability.
The situation is a little bit similar to Zeno's paradox. In a similar sense as Zeno's argument for motion not being able to occur, we could argue that the arrow can't hit any part of the dartboard because the probability to hit it is zero everywhere.
Another answer has mentioned publication bias. However, that is not really what you were asking about, which is data dredging. A pertinent XKCD illustration is:
Best Answer
Failing to look at (plot) the data.