Science is emphatically not a belief system. It doesn’t require faith, and it works: civilisation is built on science working. But it’s a full-time job to keep on top of one subject, and impossible to stay up to date across a range of fields. We have to trust that the system works. But does it?
This is the process: scientists do the research – primarily paid for by you – which gets written up and peer-reviewed before publication as a paper in a journal. Getting published in a journal is not a mark of truth but that your research is credible enough to warrant entering the literature for ongoing scrutiny. Published papers are the benchmark of academic success, and the media’s main focus.
At least that’s the theory. In practice, scientific publishing is riddled with problems. At one end there are the glamour journals, Nature, Science and a couple of others. These put out the biggest, noisiest and most exciting scientific discoveries, and careers are made if you get your research in. Are they more right than lower-ranked journals? Probably not, but they are exposed to great scrutiny under the maxim that extraordinary claims demand extraordinary evidence.
Still, plenty of guff slips through the cracks. In December 2014, Science published a high-profile paper on how gay canvassers had a more lasting positive effect than heterosexuals on voters’ opinions about same-sex marriage. By June this year the study had been retracted amid allegations of data discrepancies.
In 2010 the claim that a lifeform relying on arsenic (instead of an essential ingredient of DNA, phosphate) had been discovered in California was immediately crucified online and eventually in print for being wrong in many ways. It has never been retracted, and remains alive in one of the top journals. Since 2001, though, retractions have gone up by more than 10 times, and most are as a result of misconduct. On Tuesday a major publisher withdrew 64 papers as a result of faked peer review. The count on this charge is now more than 230.
At the other end of the scale are hundreds of journals that are perfunctory, undiscriminating or plain crap. Some are invented simply to publish bonkers ideas, such as DeNovo, seemingly in existence to claim a scientific basis for the yeti. Even biblical creationists have their own peer-reviewed journal, but, as the man says, sticking feathers up your butt does not make you a chicken. We’ve often said that, like democracy, the old model of peer review in journals is the least bad system we have. I’m beginning to wonder if it’s fit for purpose.
In the development of drugs, the system has been gamed for decades, with clinical trial data disappearing and positive results cherry-picked. Things are slowly changing as a result of dogged campaigning. Projects such as AllTrials fight to render research open, and therefore less prone to selective reporting of trials. In the US, regulation requiring researchers to register their methods before collecting data was instigated in 2000. Earlier this month a study found that in trials for heart-disease treatments, positive results went from 57% before the regulation to just 8% after. Preregistration is gaining some traction in psychology too, a field blighted by irreproducible results.
Many of us thought that journals and the concept of the paper would not have lasted into the second decade of the 21st century. The internet has rendered much of the idea behind the academic paper obsolete. In physics and astronomy, many papers are released into the wild of peer analysis long before making it into the traditional journals. Biology is just beginning to catch up.
Scrutiny of the science media is now widespread, as it should be. We are fed fudgings, misunderstandings, errors and fabrications every day, but things are better than they were. The media suckles on PR dressed as research, such as stories that redheads are going extinct (they are not, nor can they) but rebuttals are quick and robust. There has been an explosion of writers and scientists, primarily online, not in the traditional media, eager to scrutinise research. But, with equal volume, disinformation about climate change, vaccination, creationism also abounds. I don’t know how a non-expert can navigate this quagmire.
The traditional form of the book is also a problem. The days are long gone when new research was unveiled in books, the high point being The Origin of Species. But we fetishise books, and pedlars of waffly guff, such as the pharmacologist Susan Greenfield, use them to broadcast their opinions – masquerading as science. Ah, but my “book is based on 250 peer-reviewed papers” said Lady Greenfield in a debate with clinical psychologist Dr Vaughan Bell on Channel 4 News this month, about her longstanding media-wooing claims about the brain-damaging effects of technology. That number, actually 234, had gone down from Greenfield’s original claim of it being 500 in an earlier BBC interview, following autism expert Professor Dorothy Bishop’s interrogation of those references, which also points out that the relevance of some of these to Greenfield’s claims is tenuous. Out of these two Oxford professors, whom can we trust? Here’s a clue: it’s not Greenfield.
Science is complex, hard and important, but it is a system of discovery that is riddled with problems. So can we trust it? Just, I think, for now, but the need for reform is profound. Trust it as long as you know that we urgently need to make it work better.