Relationship between geographical location and evaluation of developer contributions in github | Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement
Article No.: 22, Pages 1 - 8
Abstract
Background Open source software projects show gender bias suggesting that other demographic characteristics of developers, like geographical location, can negatively influence evaluation of contributions too. Aim This study contributes to this emerging body of knowledge in software development by presenting a quantitative analysis of the relationship between the geographical location of developers and evaluation of their contributions on GitHub. Method We present an analysis of 70,000+ pull requests selected from 17 most actively participating countries to model the relationship between the geographical location of developers and pull request acceptance decision. Results and Conclusion We observed structural differences in pull request acceptance rates across 17 countries. Countries with no apparent similarities such as Switzerland and Japan had one of the highest pull request acceptance rates while countries like China and Germany had one of the lowest pull request acceptance rates. Notably, higher acceptance rates were observed for all but one country when pull requests were evaluated by developers from the same country.
References
[1]
2017. Bitbucket. https://bitbucket.org/
[2]
2017. GHTorrent. http://ghtorrent.org/downloads.html.
[3]
2017. GitHub, Inc. https://github.com/
[4]
2017. Stack Exchange. https://stackexchange.com/
[5]
2017. Stack Overflow. https://stackoverflow.com/
[6]
Victor R Basili, Forrest Shull, and Filippo Lanubile. 1999. Building knowledge through families of experiments. IEEE Transactions on Software Engineering 25, 4 (1999), 456--473.
[7]
Christian Bird, Alex Gourley, Prem Devanbu, Anand Swaminathan, and Greta Hsu. 2007. Open borders? immigration in open source projects. In Mining Software Repositories, 2007. ICSE Workshops MSR'07. Fourth International Workshop on. IEEE, 6--6.
[8]
Jacob Cohen, Patricia Cohen, Stephen G West, and Leona S Aiken. 2013. Applied multiple regression/correlation analysis for the behavioral sciences. Routledge.
[9]
Harald Cramér. 1999. Mathematical methods of statistics. Vol. 9. Princeton university press.
[10]
Shane Curcuru. 2015. 3 key elements that define every open source. https://opensource.com/life/15/2/3-key-elements-every-open-source-project.
[11]
Georgios Gousios, Martin Pinzger, and Arie van Deursen. 2014. An exploratory study of the pull-based software development model. In Proceedings of the 36th International Conference on Software Engineering. ACM, 345--355.
[12]
Georgios Gousios, Margaret-Anne Storey, and Alberto Bacchelli. 2016. Work practices and challenges in pull-based development: The contributor's perspective. In Software Engineering (ICSE), 2016 IEEE/ACM 38th International Conference on. IEEE, 285--296.
[13]
Georgios Gousios, Margaret-Anne Storey, and Alberto Bacchelli. 2016. Work Practices and Challenges in Pull-Based Development: The Contributor's Perspective. In Proceedings of the 38th International Conference on Software Engineering (ICSE).
[14]
Georgios Gousios and Andy Zaidman. 2014. A Dataset for Pull-based Development Research. In Proceedings of the 11th Working Conference on Mining Software Repositories (MSR 2014). ACM, New York, NY, USA, 368--371.
[15]
Marek Hlavac. 2015. stargazer: Well-Formatted Regression and Summary Statistics Tables. http://CRAN.R-project.org/package=stargazer R package version 5.2.
[16]
Jennifer Marlow, Laura Dabbish, and Jim Herbsleb. 2013. Impression formation in online peer production: activity traces and personal profiles in github. In Proceedings of the 2013 conference on Computer supported cooperative work. ACM, 117--128.
[17]
Nora McDonald and Sean Goggins. 2013. Performance and participation in open source software on github. In CHI'13 Extended Abstracts on Human Factors in Computing Systems. ACM, 139--144.
[18]
Gianmarco Paris, Giulio De Leo, Paolo Menozzi, and Marino Gatto. 1998. Region-based citation bias in science. Nature 396, 6708 (1998), 210.
[19]
Raphael Pham, Leif Singer, Olga Liskin, Fernando Figueira Filho, and Klaus Schneider. 2013. Creating a shared understanding of testing culture on a social coding site. In Software Engineering (ICSE), 2013 35th International Conference on. IEEE, 112--121.
[20]
R Core Team. 2015. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/
[21]
Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu. 2014. A large scale study of programming languages and code quality in github. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 155--165.
[22]
Anna Sandberg. 2015. Competing biases: Effects of gender and nationality in sports judging. (2015).
[23]
Walt Scacchi. 2007. Free/open source software development: Recent research results and methods. Advances in Computers 69 (2007), 243--295.
[24]
Daricélio Moreira Soares, Manoel Limeira de Lima Júnior, Leonardo Murta, and Alexandre Plastino. 2015. Acceptance factors of pull requests in open-source projects. In Proceedings of the 30th Annual ACM Symposium on Applied Computing. ACM, 1541--1546.
[25]
Josh Terrell, Andrew Kofink, Justin Middleton, Clarissa Rainear, Emerson Murphy-Hill, and Chris Parnin. 2016. Gender bias in open source: Pull request acceptance of women versus men. Technical Report. PeerJ PrePrints.
[26]
Jason Tsay, Laura Dabbish, and James Herbsleb. 2014. Influence of social and technical factors for evaluating contribution in GitHub. In Proceedings of the 36th international conference on Software engineering. ACM, 356--366.
[27]
Bogdan Vasilescu, Vladimir Filkov, and Alexander Serebrenik. 2015. Perceptions of diversity on GitHub: A user survey. CHASE. IEEE (2015).
[28]
Bogdan Vasilescu, Daryl Posnett, Baishakhi Ray, Mark GJ van den Brand, Alexander Serebrenik, Premkumar Devanbu, and Vladimir Filkov. 2015. Gender and tenure diversity in GitHub teams. In CHI. ACM.
[29]
Quang H Vuong. 1989. Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica: Journal of the Econometric Society (1989), 307--333.
[30]
Peter Weißgerber, Daniel Neu, and Stephan Diehl. 2008. Small patches get in!. In Proceedings of the 2008 international working conference on Mining software repositories. ACM, 67--76.
[31]
Y. Yu, H. Wang, V. Filkov, P. Devanbu, and B. Vasilescu. 2015. Wait for It: Determinants of Pull Request Evaluation Latency on GitHub. In 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories. 367--371.
Information & Contributors
Information
Published In
ESEM '18: Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement
October 2018
487 pages
Copyright © 2018 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- IEEE CS
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 October 2018
Permissions
Request permissions for this article.
Check for updates
Badges
Best Dataset
Author Tags
Qualifiers
- Research-article
Conference
Acceptance Rates
Overall Acceptance Rate 130 of 594 submissions, 22%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- Downloads (Last 12 months)78
- Downloads (Last 6 weeks)5
Reflects downloads up to 19 Feb 2025
Other Metrics
Citations
- de Souza CMa EWong JYoon DBeschastnikh I(2024)Revealing Software Development Work Patterns with PR-Issue Graph TopologiesProceedings of the ACM on Software Engineering10.1145/36608131:FSE(2402-2423)Online publication date: 12-Jul-2024
- Kola-Olawuyi AWeeraddana NNagappan MSpinellis DConstantinou EBacchelli A(2024)The Impact of Code Ownership of DevOps Artefacts on the Outcome of DevOps CI BuildsProceedings of the 21st International Conference on Mining Software Repositories10.1145/3643991.3644924(543-555)Online publication date: 15-Apr-2024
- Khatoonabadi SAbdellatif ACosta DShihab E(2024)Predicting the First Response Latency of Maintainers and Contributors in Pull RequestsIEEE Transactions on Software Engineering10.1109/TSE.2024.344374150:10(2529-2543)Online publication date: 1-Oct-2024
- Nakano TShimari KKula RTreude CCheong MMatsumoto K(2024)Nigerian Software Engineer or American Data Scientist? GitHub Profile Recruitment Bias in Large Language Models2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58944.2024.00063(624-629)Online publication date: 6-Oct-2024
- Zhou YSun YWu JZhang JHuang G(2024)Wiki2GH: A Recommendation Service to Link Software Engineering Knowledge to Practical DevelopmentService Science10.1007/978-981-97-5760-2_14(203-220)Online publication date: 19-Aug-2024
- AlHauli AAlMarzouq M(2024)National Financial Status as a Predictor of Innovation in Crowdsourcing Platforms: The Case of GitHubNavigating the Technological Tide: The Evolution and Challenges of Business Model Innovation10.1007/978-3-031-67431-0_25(261-268)Online publication date: 1-Aug-2024
- Rastogi A(2024)Roads Ahead to Diversity and Inclusion by Software EngineeringEquity, Diversity, and Inclusion in Software Engineering10.1007/978-1-4842-9651-6_1(3-16)Online publication date: 21-Sep-2024
- Khatoonabadi SCosta DMujahid SShihab E(2023)Understanding the Helpfulness of Stale Bot for Pull-Based Development: An Empirical Study of 20 Large Open-Source ProjectsACM Transactions on Software Engineering and Methodology10.1145/362473933:2(1-43)Online publication date: 23-Dec-2023
- Maddila CUpadrasta SBansal CNagappan NGousios Gvan Deursen A(2023)Nudge: Accelerating Overdue Pull Requests toward CompletionACM Transactions on Software Engineering and Methodology10.1145/354479132:2(1-30)Online publication date: 30-Mar-2023
- Khatoonabadi SCosta DAbdalkareem RShihab E(2023)On Wasted Contributions: Understanding the Dynamics of Contributor-Abandoned Pull Requests–A Mixed-Methods Study of 10 Large Open-Source ProjectsACM Transactions on Software Engineering and Methodology10.1145/353078532:1(1-39)Online publication date: 13-Feb-2023
- Show More Cited By
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.