We base many things on opinion or belief—from religion to politics to daily habits to relationships. However, science is the one thing that should escape any form of bias. The fact that science is based on physical laws means that it provides a disciplined, consistent method to explain events of nature. The scientific method (systematic observation, measurement, experiment, and the formulation, testing, and modification of hypotheses) has been established to ensure the most accurate knowledge based on empirical data.
And yet, according to a new meta-analysis, even the strictest scientific methods aren’t enough to avoid bias in all its forms.
The PLOS journal published their review of 35 reports, which they analyzed for spin. The definition of spin is “reporting practices that distort the interpretation of results and mislead readers so that results are viewed in a more favorable light.”
It’s not biased in the research itself, but in the presentation or communication of the outcomes of the research. Spin is dangerous, as it may “negatively impact the development of further studies, clinical practice, and health policies.”
All but one of the 35 reports analyzed have been published in the last ten years, with 22 being published in the last five years. These reports all looked at the prevalence of spin in published literature — from randomized controlled trials to surveys to regulatory and company documents.
The fields of study of the literature examined were varied but mainly focused on biomedical interventions.
The researchers defined spin as any one of four ways:
- Reporting practices that distort the interpretation of results and create misleading conclusions, suggesting a more favorable result
- Discordance between results and their interpretation, with the interpretation being more favorable than the results
- Attribution of causality when study design does not allow for it
- Overinterpretation or inappropriate extrapolation of results
Promoting Open Data and No Spin in Research
- 100% of texts regarding a sample of implantable defibrillator trials featured rhetorical practices that led to spin
- Even in the lowest prevalence, 9.7% of trials in systemic therapy in lung cancer presented conclusions that were discordant from the results of the study
- The abstracts of 30% of trials with non-significant results had “high levels of spin” included in the conclusions
- The main texts of 22% of trials with non-significant results had “high levels of spin” included in the conclusions
Pretty scary to think that science, the impartial discipline, can still be influenced by the human factor, right? But what can be done to prevent this sort of spin and keep the science as unbiased as possible?
The researchers had some suggestions:
- “First, as routinely occurs, peer reviewers and journal editors check that abstract and manuscript conclusions are consistent with the study results, for inappropriate use of causal language, and for overgeneralization.”
- “Second, clinical practice and public health guidelines should be developed based on systematic reviews to ensure that recommendations are founded on rigorous data and not misleading conclusions.”
- “Third, promoting fully open data or inviting published interpretation of published data from multiple researchers could mitigate the occurrence of spin.”
- “Finally, structural reforms within academia are needed to change research incentives and reward structures that emphasize ‘positive’ conclusions, including the pressure to publish and media attention.”
