[Math] Replication crisis in mathematics

proof-theoryresearch

Lately, I have been learning about the replication crisis, see How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth (good YouTube video) — by Michael Shermer and Stuart Ritchie. According to Wikipedia, the replication crisis (also known as the replicability crisis or reproducibility crisis) is

an ongoing methodological crisis in which it has been found that many
scientific studies are difficult or impossible to replicate or
reproduce. The replication crisis affects the social sciences and
medicine most severely.

Has the replication crisis impacted (pure) mathematics, or is mathematics unaffected? How should results in mathematics be reproduced? How can complicated proofs be replicated, given that so few people are able to understand them to begin with?

Best Answer

Mathematics does have its own version of the replicability problem, but for various reasons it is not as severe as in some scientific literature.

A good example is the classification of finite simple groups - this was a monumental achievement (mostly) completed in the 1980's, spanning tens of thousands of pages written by dozens of authors. But over the past 20 years there has been significant ongoing effort undertaken by Gorenstein, Lyons, Solomon, and others to consolidate the proof in one place. This is partially to simplify and iron out kinks in the proof, but also out of a very real concern that the proof will be lost as experts retire and the field attracts fewer and fewer new researchers. This is one replicability issue in mathematics: some bodies of mathematical knowledge slide into folklore or arcana unless there is a concerted effort by the next generation to organize and preserve them.

Another example is the ongoing saga of Mochizuki's proposed proof of the abc conjecture. The proof involve thousands of pages of work that remains obscure to all but a few, and there remains serious disagreement over whether the argument is correct. There are numerous other examples where important results are called into question because few experts spend the time and energy necessary to carefully work through difficult foundational theory - symplectic geometry provides another recent example.

Why do I think these issues are not as big of a problem for mathematics as analogous issues in the sciences?

  1. Negative results: If you set out to solve an important mathematical problem but instead discover a disproof or counterexample, this is often just as highly valued as a proof. This provides a check against the perverse incentives which motivate some empirical researchers to stretch their evidence for the sake of getting a publication.
  2. Interconnectedness: Most mathematical research is part of an ecosystem of similar results about similar objects, and in an area with enough activity it is difficult for inconsistencies to develop and persist unnoticed.
  3. Generalization: Whenever there is a major mathematical breakthrough it is normally followed by a flurry of activity to extend it and solve other related problems. This entails not just replicating the breakthrough but clarifying it and probing its limits - a good example of this is all the work in the Langlands program which extends and clarifies Wiles' work on the modularity theorem.
  4. Purity: social science and psychology research is hard because the results of an experiment depend on norms and empirical circumstances which can change significantly over time - for instance, many studies about media consumption before the 90's were rendered almost irrelevant by the internet. The foundations of an area of mathematics can change, but the logical correctness of a mathematical argument can't (more or less).
Related Question