The Economist complains that modern scientists are doing “too much trusting and not enough verifying—to the detriment of the whole of science, and of humanity”:
Academic scientists readily acknowledge that they often get things wrong. But they also hold fast to the idea that these errors get corrected over time as other scientists try to take the work further. Evidence that many more dodgy results are published than are subsequently corrected or withdrawn calls that much-vaunted capacity for self-correction into question. There are errors in a lot more of the scientific papers being published, written about and acted on than anyone would normally suppose, or like to think.
Various factors contribute to the problem. Statistical mistakes are widespread. The peer reviewers who evaluate papers before journals commit to publishing them are much worse at spotting mistakes than they or others appreciate. Professional pressure, competition and ambition push scientists to publish more quickly than would be wise. A career structure which lays great stress on publishing copious papers exacerbates all these problems. “There is no cost to getting things wrong,” says Brian Nosek, a psychologist at the University of Virginia who has taken an interest in his discipline’s persistent errors. “The cost is not getting them published.”
In a separate piece, the magazine suggests some steps for the scientific community to reclaim legitimacy:
A start would be getting to grips with statistics, especially in the growing number of fields that sift through untold oodles of data looking for patterns. Geneticists have done this, and turned an early torrent of specious results from genome sequencing into a trickle of truly significant ones. Ideally, research protocols should be registered in advance and monitored in virtual notebooks. This would curb the temptation to fiddle with the experiment’s design midstream so as to make the results look more substantial than they are. (It is already meant to happen in clinical trials of drugs, but compliance is patchy.) Where possible, trial data also should be open for other researchers to inspect and test.
Jerry Coyne reacts:
The main lesson of these pieces is that we shouldn’t trust a scientific result unless it’s been independently replicated—preferably more than once. That’s something we should already know, but what we don’t know is how many findings—and the articles deal largely with biomedical research—haven’t been replicable, how many others haven’t even been subject to replication, and how shoddy the reviewing process is, so that even a published result may be dubious.
As I read these pieces, I did so captiously, really wanting to find some flaws with their conclusions. I don’t like to think that there are so many problems with my profession. But the authors have done their homework and present a pretty convincing case that science, especially given the fierce competition to get jobs and succeed in them, is not doing a bang-up job. That doesn’t mean it is completely flawed, for if that were true we’d make no advances at all, and we do know that many discoveries in recent years (dinosaurs evolving into birds, the Higgs boson, black matter, DNA sequences, and so on) seem solid.
Relatedly, Michael White investigates the motivations of cheating scientists:
The trouble happens when good ideas turn out to be wrong. Based on a compelling idea, a scientist may have invested months or years of work, given presentations to colleagues or even the public, and published promising results in a high-profile journal. But further experiments don’t turn out as expected, and it becomes clear that the cherished idea that was too good not to be true now needs to be abandoned. At this point, as physicist Bob Park described it in Voodoo Science: The Road From Foolishness to Fraud, scientists reach a fork in the road. “In one direction lies the admission that they may have been mistaken. … In the other direction is denial. … Few if any scientists are so clever or so lucky that they will not come to such a fork in their career.”
The Dish 