A manifesto for reproducible science - PubMed
- ️Sun Jan 01 2017
A manifesto for reproducible science
Marcus R Munafò et al. Nat Hum Behav. 2017.
Abstract
Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.
Conflict of interest statement
Competing interests M.R.M, together with C.D.C and D.V.M.B., has received funding from the BBSRC (grant number BB/N019660/1) to convene a workshop on advanced methods for reproducible science, and is chair of the CHDI Foundation Independent Statistical Standing Committee. B.A.N. is executive director of the non-profit Center for Open Science with a mission to increase openness, integrity and reproducibility of research. N.P.d.S. leads the NC3Rs programme of work on experimental design, which developed the ARRIVE guidelines and Experimental Design Assistant. J.J.W. is director, experimental design, at CHDI Management/CHDI Foundation, a non-profit biomedical research organization exclusively dedicated to developing therapeutics for Huntington’s disease. The other authors declare no competing interests.
Figures

An idealized version of the hypothetico-deductive model of the scientific method is shown. Various potential threats to this model exist (indicated in red), including lack of replication, hypothesizing after the results are known (HARKing), poor study design, low statistical power, analytical flexibility, P-hacking, publication bias and lack of data sharing. Together these will serve to undermine the robustness of published research, and may also impact on the ability of science to self-correct.

In January 2014, the journal Psychological Science (PSCI) introduced badges for articles with open data. Immediately afterwards, the proportion of articles with open data increased steeply, and by October 2015, 38% of articles in Psychological Science had open data. For comparison journals (Clinical Psychological Science (CPS), Developmental Psychology (DP), Journal of Experimental Psychology: Learning, Memory and Cognition (JEPLMC) and Journal of Personality and Social Psychology (JPSP)) the proportion of articles with open data remained uniformly low. Figure adapted from ref. , PLoS.
Similar articles
-
Depressing time: Waiting, melancholia, and the psychoanalytic practice of care.
Salisbury L, Baraitser L. Salisbury L, et al. In: Kirtsoglou E, Simpson B, editors. The Time of Anthropology: Studies of Contemporary Chronopolitics. Abingdon: Routledge; 2020. Chapter 5. In: Kirtsoglou E, Simpson B, editors. The Time of Anthropology: Studies of Contemporary Chronopolitics. Abingdon: Routledge; 2020. Chapter 5. PMID: 36137063 Free Books & Documents. Review.
-
Using Experience Sampling Methodology to Capture Disclosure Opportunities for Autistic Adults.
Love AMA, Edwards C, Cai RY, Gibbs V. Love AMA, et al. Autism Adulthood. 2023 Dec 1;5(4):389-400. doi: 10.1089/aut.2022.0090. Epub 2023 Dec 12. Autism Adulthood. 2023. PMID: 38116059 Free PMC article.
-
How to create accessible research summaries for the developmental language disorder community.
Gasparini L, Ziegenfusz S, Turner N, Leitão S, St Clair MC, Jackson E. Gasparini L, et al. Int J Lang Commun Disord. 2025 Jan-Feb;60(1):e13142. doi: 10.1111/1460-6984.13142. Int J Lang Commun Disord. 2025. PMID: 39625399
-
Vergani M, Perry B, Freilich J, Chermak S, Scrivens R, Link R, Kleinsman D, Betts J, Iqbal M. Vergani M, et al. Campbell Syst Rev. 2024 Apr 28;20(2):e1397. doi: 10.1002/cl2.1397. eCollection 2024 Jun. Campbell Syst Rev. 2024. PMID: 38686101 Free PMC article. Review.
-
Elwenspoek MM, Thom H, Sheppard AL, Keeney E, O'Donnell R, Jackson J, Roadevin C, Dawson S, Lane D, Stubbs J, Everitt H, Watson JC, Hay AD, Gillett P, Robins G, Jones HE, Mallett S, Whiting PF. Elwenspoek MM, et al. Health Technol Assess. 2022 Oct;26(44):1-310. doi: 10.3310/ZUCE8371. Health Technol Assess. 2022. PMID: 36321689 Free PMC article.
Cited by
-
Kabanda SM, Cengiz N, Rajaratnam K, Watson BW, Brown Q, Esterhuizen TM, Moodley K. Kabanda SM, et al. S Afr J Sci. 2023 May-Jun;119(5-6):15129. doi: 10.17159/sajs.2023/15129. Epub 2023 May 30. S Afr J Sci. 2023. PMID: 39483790 Free PMC article.
-
Towards a Research Agenda for Promoting Responsible Research Practices.
Tijdink JK, Horbach SPJM, Nuijten MB, O'Neill G. Tijdink JK, et al. J Empir Res Hum Res Ethics. 2021 Oct;16(4):450-460. doi: 10.1177/15562646211018916. Epub 2021 May 26. J Empir Res Hum Res Ethics. 2021. PMID: 34037490 Free PMC article.
-
Open science saves lives: lessons from the COVID-19 pandemic.
Besançon L, Peiffer-Smadja N, Segalas C, Jiang H, Masuzzo P, Smout C, Billy E, Deforet M, Leyrat C. Besançon L, et al. BMC Med Res Methodol. 2021 Jun 5;21(1):117. doi: 10.1186/s12874-021-01304-y. BMC Med Res Methodol. 2021. PMID: 34090351 Free PMC article.
-
Nguyen MH, Le TT, Nguyen HT, Ho MT, Nguyen HTT, Vuong QH. Nguyen MH, et al. Int J Environ Res Public Health. 2021 Apr 1;18(7):3681. doi: 10.3390/ijerph18073681. Int J Environ Res Public Health. 2021. PMID: 33916123 Free PMC article.
-
Open Developmental Science: An Overview and Annotated Reading List.
Kalandadze T, Hart SA. Kalandadze T, et al. Infant Child Dev. 2024 Jan-Feb;33(1):e2334. doi: 10.1002/icd.2334. Epub 2022 May 18. Infant Child Dev. 2024. PMID: 39308897 Free PMC article.
References
Grants and funding
LinkOut - more resources
Full Text Sources
Other Literature Sources