pubmed.ncbi.nlm.nih.gov

Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs - PubMed

  • ️Sun Jan 01 2023

. 2023 Jun 6;120(23):e2215572120.

doi: 10.1073/pnas.2215572120. Epub 2023 May 30.

Anna Dreber  2   3 , Jürgen Huber  4 , Magnus Johannesson  2 , Michael Kirchler  4 , Utz Weitzel  5   6   7 , Miguel Abellán  8 , Xeniya Adayeva  9 , Fehime Ceren Ay  10   11 , Kai Barron  12 , Zachariah Berry  13 , Werner Bönte  14   15 , Katharina Brütt  16 , Muhammed Bulutay  17 , Pol Campos-Mercade  18 , Eric Cardella  19 , Maria Almudena Claassen  20 , Gert Cornelissen  21   22 , Ian G J Dawson  23 , Joyce Delnoij  24 , Elif E Demiral  25   26 , Eugen Dimant  27 , Johannes Theodor Doerflinger  28 , Malte Dold  29 , Cécile Emery  30 , Lenka Fiala  31 , Susann Fiedler  32 , Eleonora Freddi  10   11 , Tilman Fries  12 , Agata Gasiorowska  33 , Ulrich Glogowsky  34 , Paul M Gorny  35 , Jeremy David Gretton  36 , Antonia Grohmann  37   38 , Sebastian Hafenbrädl  39 , Michel Handgraaf  24   40 , Yaniv Hanoch  23 , Einav Hart  41 , Max Hennig  42 , Stanton Hudja  43 , Mandy Hütter  42 , Kyle Hyndman  44 , Konstantinos Ioannidis  45 , Ozan Isler  46 , Sabrina Jeworrek  47   48 , Daniel Jolles  49 , Marie Juanchich  49 , Raghabendra Pratap Kc  50 , Menusch Khadjavi  7   51   52 , Tamar Kugler  53 , Shuwen Li  54 , Brian Lucas  13 , Vincent Mak  55 , Mario Mechtel  8 , Christoph Merkle  37   38 , Ethan Andrew Meyers  36 , Johanna Mollerstrom  56   57 , Alexander Nesterov  9 , Levent Neyse  12   58 , Petra Nieken  35   59 , Anne-Marie Nussberger  60 , Helena Palumbo  21 , Kim Peters  30 , Angelo Pirrone  61 , Xiangdong Qin  54 , Rima Maria Rahal  62 , Holger Rau  63 , Johannes Rincke  64 , Piero Ronzani  65 , Yefim Roth  66 , Ali Seyhun Saral  67 , Jan Schmitz  6 , Florian Schneider  68 , Arthur Schram  16 , Simeon Schudy  59   69 , Maurice E Schweitzer  70 , Christiane Schwieren  71 , Irene Scopelliti  72 , Miroslav Sirota  49 , Joep Sonnemans  45 , Ivan Soraperra  45 , Lisa Spantig  73   74 , Ivo Steimanis  75 , Janina Steinmetz  72 , Sigrid Suetens  76 , Andriana Theodoropoulou  49 , Diemo Urbig  15   77 , Tobias Vorlaufer  78 , Joschka Waibel  74 , Daniel Woods  3 , Ofir Yakobi  36 , Onurcan Yilmaz  79 , Tomasz Zaleskiewicz  33 , Stefan Zeisberger  6   80 , Felix Holzmeister  3

Affiliations

Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs

Christoph Huber et al. Proc Natl Acad Sci U S A. 2023.

Abstract

Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently, it has been tested in experimental studies yielding a body of rather inconclusive empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity-variation in true effect sizes across various reasonable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. We find a small adverse effect of competition on moral behavior in a meta-analysis of the pooled data. The crowd-sourced design of our study allows for a clean identification and estimation of the variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity-estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs-indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving toward much larger data collections on various experimental designs testing the same hypothesis.

Keywords: competition; experimental design; generalizability; metascience; moral behavior.

PubMed Disclaimer

Conflict of interest statement

The authors declare no competing interest.

Figures

Fig. 1.
Fig. 1.

Forest plot of meta-analytic results. (A) Plotted are the point estimates and the 95% CIs of the effect sizes in the 45 experimental designs and a random-effects meta-analysis for analytic approach A (in Cohen’s d units). There is statistically significant evidence (P < 0.005) of a negative effect of competition on moral behavior in four of the individual designs and suggestive evidence (P < 0.05) in four additional designs, and there is statistically significant evidence (P < 0.005) of a positive effect of competition on moral behavior in one of the individual designs and suggestive evidence (P < 0.05) in one additional design. There is suggestive evidence of an adverse effect of competition on moral behavior in the meta-analysis (d = −0.085, 95% CI [−0.147, −0.022], P = 0.008). (B) Plotted are the point estimates and the 95% CIs of the effect sizes in the 45 experimental designs and a random-effects meta-analysis for analytic approach B (in Cohen’s d units). There is statistically significant evidence (P < 0.005) of a negative effect of competition on moral behavior in four of the individual designs and suggestive evidence (P < 0.05) in three additional designs, and there is statistically significant evidence (P < 0.005) of a positive effect of competition on moral behavior in one of the individual designs and suggestive evidence (P < 0.05) in one additional design. There is statistically significant evidence of an adverse effect of competition on moral behavior in the meta-analysis (d = −0.086, 95% CI [−0.144, −0.027], P = 0.004).

Fig. 2.
Fig. 2.

Relationship between effect sizes and experimental design quality. (A) Plotted are the 45 estimated effect sizes in analytic approach A over the average (demeaned) quality ratings of the experimental designs. The linear relationship between the two variables estimated using a meta-regression is also plotted together with its 95% CI, revealing no systematic relationship (b = 0.033, se = 0.033, P = 0.316; R2 = 0.000). (B) Plotted are the 45 estimated effect sizes in analytic approach B over the average (demeaned) quality ratings of the experimental designs. The linear relationship between the two variables estimated using a meta-regression is also plotted together with its 95% CI, revealing no systematic relationship (b = 0.034, se = 0.031, P = 0.269; R2 = 0.000).

Fig. 3.
Fig. 3.

Predicted meta-analytic effect sizes in different experimental design sub-groups. (A) Plotted are the predicted values and 95% CIs of the meta-analytic effect size for analytic approach A, for the different conceptualizations of moral behavior and the different operationalizations of the competition intervention. The predicted values are based on the meta-regression tabulated in

SI Appendix, Table S6

, and the prediction for each design variable is carried out at the mean of the other design variables. *P < 0.05, **P < 0.005. (B) Plotted are the predicted values and 95% CIs of the meta-analytic effect size for analytic approach B, for the different conceptualizations of moral behavior and the different operationalizations of the competition intervention. The predicted values are based on the meta-regression tabulated in

SI Appendix, Table S6

, and the prediction for each design variable is carried out at the mean of the other design variables. *P < 0.05, **P < 0.005.

Fig. 4.
Fig. 4.

Illustration of the importance of experimental design heterogeneity. Plotted is the normal density function of the effect size distribution and the associated 95% CI of conducting a single-design study randomly drawn from the 45 experimental designs (for analytic approach B, isolating design heterogeneity). The mean of the density function is equal to the (equally weighted) mean of the 45 designs m = −0.085, and the variance of the density function is defined as the estimated design heterogeneity (τ2 = 0.028) plus the average sampling variance (σ2 = 0.012) of the 45 experimental designs. The 95% CI is [−0.477, 0.308], illustrating that single-design studies yield imprecise estimates if the estimated design heterogeneity is incorporated. Plotted is also the normal density function and the 95% CI based on only the design heterogeneity (τ2), providing a lower bound of the confidence interval when the sampling variance (sample size) goes to zero (infinity), illustrating that also this lower bound results in a wide confidence interval of [−0.415, 0.246]. The intervals in Fig. 4 were not preregistered and should not be interpreted as hypothesis tests, but only as an illustration of the importance of design heterogeneity.

Similar articles

Cited by

  • Examining the replicability of online experiments selected by a decision market.

    Holzmeister F, Johannesson M, Camerer CF, Chen Y, Ho TH, Hoogeveen S, Huber J, Imai N, Imai T, Jin L, Kirchler M, Ly A, Mandl B, Manfredi D, Nave G, Nosek BA, Pfeiffer T, Sarafoglou A, Schwaiger R, Wagenmakers EJ, Waldén V, Dreber A. Holzmeister F, et al. Nat Hum Behav. 2024 Nov 19. doi: 10.1038/s41562-024-02062-9. Online ahead of print. Nat Hum Behav. 2024. PMID: 39562799

  • Can large language models help predict results from a complex behavioural science study?

    Lippert S, Dreber A, Johannesson M, Tierney W, Cyrus-Lai W, Uhlmann EL; Emotion Expression Collaboration; Pfeiffer T. Lippert S, et al. R Soc Open Sci. 2024 Sep 25;11(9):240682. doi: 10.1098/rsos.240682. eCollection 2024 Sep. R Soc Open Sci. 2024. PMID: 39323554 Free PMC article.

  • Heterogeneity in effect size estimates.

    Holzmeister F, Johannesson M, Böhm R, Dreber A, Huber J, Kirchler M. Holzmeister F, et al. Proc Natl Acad Sci U S A. 2024 Aug 6;121(32):e2403490121. doi: 10.1073/pnas.2403490121. Epub 2024 Jul 30. Proc Natl Acad Sci U S A. 2024. PMID: 39078672 Free PMC article.

References

    1. Smith A., The Wealth of Nations (Liberty Fund, Indianapolis, 1776).
    1. Smith A., “Lectures on Jurisprudence”, in The Glasgow Edition of the Works and Correspondence of Adam Smith, Meek R. L., Raphael D. D., Stein P. G., Eds. (Oxford University Press, Oxford, 1978). vol. 5.
    1. Hirschman A. O., The Passion and the Interest: Political Arguments for Capitalism Before Its Triumph (Princeton University Press, Princeton, 1977).
    1. Henrich J., et al. , In search of homo economicus: Behavioral experiments in 15 small-scale societies. Am. Econ. Rev. 91, 73–78 (2001).
    1. Henrich J., et al. , Costly punishment across human societies. Science 312, 1767–1770 (2006). - PubMed

LinkOut - more resources