A manifesto for reproducible science - PubMed
- ️Sun Jan 01 2017
A manifesto for reproducible science
Marcus R Munafò et al. Nat Hum Behav. 2017.
Abstract
Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.
Conflict of interest statement
Competing interests M.R.M, together with C.D.C and D.V.M.B., has received funding from the BBSRC (grant number BB/N019660/1) to convene a workshop on advanced methods for reproducible science, and is chair of the CHDI Foundation Independent Statistical Standing Committee. B.A.N. is executive director of the non-profit Center for Open Science with a mission to increase openness, integrity and reproducibility of research. N.P.d.S. leads the NC3Rs programme of work on experimental design, which developed the ARRIVE guidelines and Experimental Design Assistant. J.J.W. is director, experimental design, at CHDI Management/CHDI Foundation, a non-profit biomedical research organization exclusively dedicated to developing therapeutics for Huntington’s disease. The other authors declare no competing interests.
Figures

An idealized version of the hypothetico-deductive model of the scientific method is shown. Various potential threats to this model exist (indicated in red), including lack of replication, hypothesizing after the results are known (HARKing), poor study design, low statistical power, analytical flexibility, P-hacking, publication bias and lack of data sharing. Together these will serve to undermine the robustness of published research, and may also impact on the ability of science to self-correct.

In January 2014, the journal Psychological Science (PSCI) introduced badges for articles with open data. Immediately afterwards, the proportion of articles with open data increased steeply, and by October 2015, 38% of articles in Psychological Science had open data. For comparison journals (Clinical Psychological Science (CPS), Developmental Psychology (DP), Journal of Experimental Psychology: Learning, Memory and Cognition (JEPLMC) and Journal of Personality and Social Psychology (JPSP)) the proportion of articles with open data remained uniformly low. Figure adapted from ref. , PLoS.
Similar articles
-
Depressing time: Waiting, melancholia, and the psychoanalytic practice of care.
Salisbury L, Baraitser L. Salisbury L, et al. In: Kirtsoglou E, Simpson B, editors. The Time of Anthropology: Studies of Contemporary Chronopolitics. Abingdon: Routledge; 2020. Chapter 5. In: Kirtsoglou E, Simpson B, editors. The Time of Anthropology: Studies of Contemporary Chronopolitics. Abingdon: Routledge; 2020. Chapter 5. PMID: 36137063 Free Books & Documents. Review.
-
Using Experience Sampling Methodology to Capture Disclosure Opportunities for Autistic Adults.
Love AMA, Edwards C, Cai RY, Gibbs V. Love AMA, et al. Autism Adulthood. 2023 Dec 1;5(4):389-400. doi: 10.1089/aut.2022.0090. Epub 2023 Dec 12. Autism Adulthood. 2023. PMID: 38116059 Free PMC article.
-
How to create accessible research summaries for the developmental language disorder community.
Gasparini L, Ziegenfusz S, Turner N, Leitão S, St Clair MC, Jackson E. Gasparini L, et al. Int J Lang Commun Disord. 2025 Jan-Feb;60(1):e13142. doi: 10.1111/1460-6984.13142. Int J Lang Commun Disord. 2025. PMID: 39625399
-
Vergani M, Perry B, Freilich J, Chermak S, Scrivens R, Link R, Kleinsman D, Betts J, Iqbal M. Vergani M, et al. Campbell Syst Rev. 2024 Apr 28;20(2):e1397. doi: 10.1002/cl2.1397. eCollection 2024 Jun. Campbell Syst Rev. 2024. PMID: 38686101 Free PMC article. Review.
-
Elwenspoek MM, Thom H, Sheppard AL, Keeney E, O'Donnell R, Jackson J, Roadevin C, Dawson S, Lane D, Stubbs J, Everitt H, Watson JC, Hay AD, Gillett P, Robins G, Jones HE, Mallett S, Whiting PF. Elwenspoek MM, et al. Health Technol Assess. 2022 Oct;26(44):1-310. doi: 10.3310/ZUCE8371. Health Technol Assess. 2022. PMID: 36321689 Free PMC article.
Cited by
-
Perszyk RE, Yip MC, McConnell OL, Wang ET, Jenkins A, Traynelis SF, Forest CR. Perszyk RE, et al. Mol Pharmacol. 2021 Jul;100(1):73-82. doi: 10.1124/molpharm.120.000195. Epub 2021 May 6. Mol Pharmacol. 2021. PMID: 33958481 Free PMC article.
-
Evidence of HARKing in mouse behavioural tests of anxiety.
Rosso M, Herrera A, Würbel H, Voelkl B. Rosso M, et al. R Soc Open Sci. 2024 Aug 21;11(8):231744. doi: 10.1098/rsos.231744. eCollection 2024 Aug. R Soc Open Sci. 2024. PMID: 39169968 Free PMC article.
-
Nieto I, Navas JF, Vázquez C. Nieto I, et al. Brain Behav Immun Health. 2020 Aug;7:100123. doi: 10.1016/j.bbih.2020.100123. Epub 2020 Aug 2. Brain Behav Immun Health. 2020. PMID: 32835299 Free PMC article.
-
Video game play is positively correlated with well-being.
Johannes N, Vuorre M, Przybylski AK. Johannes N, et al. R Soc Open Sci. 2021 Feb 17;8(2):202049. doi: 10.1098/rsos.202049. R Soc Open Sci. 2021. PMID: 33972879 Free PMC article.
-
Understanding experiments and research practices for reproducibility: an exploratory study.
Samuel S, König-Ries B. Samuel S, et al. PeerJ. 2021 Apr 21;9:e11140. doi: 10.7717/peerj.11140. eCollection 2021. PeerJ. 2021. PMID: 33976964 Free PMC article.
References
Grants and funding
LinkOut - more resources
Full Text Sources
Other Literature Sources