MONDAY, Sept. 11, 2017 (HealthDay News) — Take with a grain of salt any flashy reports from clinical trials boasting groundbreaking results, a new scientific review warns.
A majority of clinical trials published in medical journals hype their findings, presenting them in a way that makes them look more favorable than they actually are, said senior researcher Lisa Bero.
An evidence review conducted by Bero and her colleagues found more than half of clinical trials “spin” their results either in the abstracts (summaries) or the main text of the published report.
“‘Breakthrough’ studies are quite rare, so any study making bold claims about [effectiveness] or causality should be viewed skeptically,” said Bero, who’s chair of medicines use and health outcomes with the University of Sydney in Australia.
“Science is an accumulation of knowledge, so new studies need to be put in the context of others,” she added.
Spin or science hype involves different strategies, Bero said. These can include:
- Claiming that statistically non-significant results actually mean something.
- Making recommendations for clinical practice that aren’t supported by study results.
- Attributing cause-and-effect relationships that aren’t supported by the findings.
- Selectively reporting only the results that support the desired conclusion.
- Presenting data in an overly favorable or optimistic light.
A hyped report might “conclude that a new treatment is beneficial when none of the results are statistically significant, or to suggest that a treatment or diagnostic test should be widely used based on the findings of a single, small study,” Bero said.
To see how deeply such hype has infiltrated medical literature, Bero and her team reviewed 35 academic studies that individually attempted to assess spin in different fields of research.
Overall, the studies found that around 56 percent of clinical trials tended to hype their results. The presence of spin ranged from about 34 percent of reports from randomized clinical trials — considered the gold standard of research — up to 84 percent of papers written about non-randomized trials.
The researchers also found hype in 86 percent of papers on observational studies. These studies do not involve actual experiments, but instead try to discover new information by observing people’s everyday lives.
Spin also was present in 26 percent of evidence reviews that attempt to sum up current medical knowledge by analyzing all the existing research, the findings showed.
Annette Flanagin is executive managing editor and vice president of operations for the Journal of the American Medical Association and the JAMA Network of publications. She said that this new analysis “provides more important information on the prevalence of biomedical studies with spin and describes the types of spin that are misleading and potentially deceptive.”
Flanagin agreed with the authors of the new report that “editors and peer reviewers should be aware of the prevalence of spin and find ways to identify and remove it from papers before publication.”
The spin produced by researchers seems motivated mainly by the desire to get a paper published by a medical journal, Bero said.
“Studies with favorable findings are more likely to get submitted for publication, get attention from other researchers, clinicians, and even the press,” Bero said.
Her team found no link between hype in studies and whether they were funded by drug or medical device companies.
Readers can protect themselves against hyperbole by comparing any study’s actual results against what the authors claim those results show, Bero noted.
“When reading a scientific paper, focus on the first paragraph of the discussion, which should simply summarize the results,” Bero said. “The rest of the discussion may make bolder statements. Beware of adjectives!”
The new analysis was published in the Sept. 11 issue of PLOS Biology.
More information
For tips on reading scientific research papers, visit the American Association for the Advancement of Science.
