pubmed.ncbi.nlm.nih.gov

A manifesto for reproducible science - PubMed

  • ️Sun Jan 01 2017

A manifesto for reproducible science

Marcus R Munafò et al. Nat Hum Behav. 2017.

Abstract

Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.

PubMed Disclaimer

Conflict of interest statement

Competing interests M.R.M, together with C.D.C and D.V.M.B., has received funding from the BBSRC (grant number BB/N019660/1) to convene a workshop on advanced methods for reproducible science, and is chair of the CHDI Foundation Independent Statistical Standing Committee. B.A.N. is executive director of the non-profit Center for Open Science with a mission to increase openness, integrity and reproducibility of research. N.P.d.S. leads the NC3Rs programme of work on experimental design, which developed the ARRIVE guidelines and Experimental Design Assistant. J.J.W. is director, experimental design, at CHDI Management/CHDI Foundation, a non-profit biomedical research organization exclusively dedicated to developing therapeutics for Huntington’s disease. The other authors declare no competing interests.

Figures

Figure 1
Figure 1. Threats to reproducible science.

An idealized version of the hypothetico-deductive model of the scientific method is shown. Various potential threats to this model exist (indicated in red), including lack of replication, hypothesizing after the results are known (HARKing), poor study design, low statistical power, analytical flexibility, P-hacking, publication bias and lack of data sharing. Together these will serve to undermine the robustness of published research, and may also impact on the ability of science to self-correct.

Figure 2
Figure 2. The impact of introducing badges for data sharing.

In January 2014, the journal Psychological Science (PSCI) introduced badges for articles with open data. Immediately afterwards, the proportion of articles with open data increased steeply, and by October 2015, 38% of articles in Psychological Science had open data. For comparison journals (Clinical Psychological Science (CPS), Developmental Psychology (DP), Journal of Experimental Psychology: Learning, Memory and Cognition (JEPLMC) and Journal of Personality and Social Psychology (JPSP)) the proportion of articles with open data remained uniformly low. Figure adapted from ref. , PLoS.

Similar articles

Cited by

References

    1. Ioannidis John P. A. Why Most Published Research Findings Are False. PLoS Medicine. 2005;2(8):e124. doi: 10.1371/journal.pmed.0020124. - DOI - PMC - PubMed
    1. Button KS. Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 2013;14:365–376. doi: 10.1038/nrn3475. - DOI - PubMed
    1. Fanelli Daniele. “Positive” Results Increase Down the Hierarchy of the Sciences. PLoS ONE. 2010;5(4):e10068. doi: 10.1371/journal.pone.0010068. - DOI - PMC - PubMed
    1. John LK, Loewenstein G, Prelec D. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol. Sci. 2012;23:524–532. doi: 10.1177/0956797611430953. - DOI - PubMed
    1. Makel MC, Plucker JA, Hegarty B. Replications in psychology research: how often do they really occur? Perspect. Psychol. Sci. 2012;7:537–542. doi: 10.1177/1745691612460688. - DOI - PubMed

LinkOut - more resources