Factors influencing code review processes in industry | Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering
Abstract
Code review is known to be an efficient quality assurance technique. Many software companies today use it, usually with a process similar to the patch review process in open source software development. However, there is still a large fraction of companies performing almost no code reviews at all. And the companies that do code reviews have a lot of variation in the details of their processes. For researchers trying to improve the use of code reviews in industry, it is important to know the reasons for these process variations. We have performed a grounded theory study to clarify process variations and their rationales. The study is based on interviews with software development professionals from 19 companies. These interviews provided insights into the reasons and influencing factors behind the adoption or non-adoption of code reviews as a whole as well as for different process variations. We have condensed these findings into seven hypotheses and a classification of the influencing factors. Our results show the importance of cultural and social issues for review adoption. They trace many process variations to differences in development context and in desired review effects.
References
[1]
S. Adolph, W. Hall, and P. Kruchten. Using grounded theory to study the experience of software development. Empirical Software Engineering, 16(4):487–513, 2011.
[2]
A. Bacchelli and C. Bird. Expectations, outcomes, and challenges of modern code review. In Proceedings of the 2013 International Conference on Software Engineering, pages 712–721. IEEE Press, 2013.
[3]
R. A. Baker Jr. Code reviews enhance software quality. In Proceedings of the 19th International Conference on Software Engineering, pages 570–571. ACM, 1997.
[4]
T. Baum, O. Liskin, K. Niklas, and K. Schneider. A faceted classification scheme for change-based industrial code review processes. In Software Quality, Reliability and Security (QRS), 2016 IEEE International Conference on. IEEE, 2016.
[5]
M. Bernhart and T. Grechenig. On the understanding of programs with continuous code reviews. In Program Comprehension (ICPC), 2013 IEEE 21st International Conference on, pages 192–198. IEEE, 2013.
[6]
K. Charmaz. Constructing Grounded Theory. Introducing Qualitative Methods series. SAGE Publications, 2014.
[7]
M. Ciolkowski, O. Laitenberger, and S. Biffl. Software reviews: The state of the practice. IEEE software, 20(6):46–51, 2003.
[8]
P. Clarke and R. V. O’Connor. The situational factors that affect the software development process: Towards a comprehensive reference framework. Information and Software Technology, 54(5):433–447, 2012.
[9]
G. Coleman and R. O’Connor. Using grounded theory to understand software process improvement: A study of irish software product companies. Information and Software Technology, 49(6):654–667, 2007.
[10]
G. Coleman and R. V. O’Connor. An investigation into software development process formation in software start-ups. Journal of Enterprise Information Management, 21(6):633–648, 2008.
[11]
J. Corbin and A. Strauss. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. SAGE Publications, 3e edition, 2007.
[12]
C. Denger and F. Shull. A practical approach for quality-driven inspections. Software, IEEE, 24(2):79–86, 2007.
[13]
M. Fagan. Design and code inspections to reduce errors in program development. IBM Systems Journal, 15(3):182–211, 1976.
[14]
M. Fagan. A history of software inspections. In Software pioneers, pages 562–573. Springer, 2002.
[15]
S. Friese. Qualitative Data Analysis with ATLAS.ti. SAGE Publications, 2012.
[16]
B. Glaser and A. Strauss. The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine, 1967.
[17]
P. Green II, T. Menzies, S. Williams, and O. El-Rawas. Understanding the value of software engineering technologies. In Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering, pages 52–61. IEEE Computer Society, 2009.
[18]
L. Harjumaa, I. Tervonen, and A. Huttunen. Peer reviews in real life-motivators and demotivators. In Quality Software, 2005.(QSIC 2005). Fifth International Conference on, pages 29–36. IEEE, 2005.
[19]
J. Iisakka and I. Tervonen. The darker side of inspection. In Proceedings of Workshop on Inspection in Software Engineering (WISE 2001), Paris, 2001.
[20]
P. Jalote and M. Haragopal. Overcoming the nah syndrome for inspection deployment. In Proceedings of the 20th International Conference on Software Engineering, pages 371–378. IEEE Computer Society, 1998.
[21]
M. Komssi, M. Kauppinen, M. Pyhajarvi, J. Talvio, and T. Mannisto. Persuading software development teams to document inspections: success factors and challenges in practice. In Requirements Engineering Conference (RE), 2010 18th IEEE International, pages 283–288. IEEE, 2010.
[22]
L. P. W. Land and J. Higgs. Factors contributing to software quality practices-an australian case study. In Wireless Communications, Networking and Mobile Computing, 2007. WiCom 2007. International Conference on, pages 5149–5152. IEEE, 2007.
[23]
M. V. Mantyla and C. Lassenius. What types of defects are really discovered in code reviews? Software Engineering, IEEE Transactions on, 35(3):430–448, 2009.
[24]
J. G. March and H. A. Simon. Organizations. John Wiley & Sons, Inc., 1958.
[25]
E. Mustonen-Ollila and K. Lyytinen. Why organizations adopt information system process innovations: a longitudinal study using diffusion of innovation theory. Information Systems Journal, 13(3):275–297, 2003.
[26]
A. J. Onwuegbuzie and N. L. Leech. Validity and qualitative research: An oxymoron? Quality & Quantity, 41(2):233–249, 2007.
[27]
W. J. Orlikowski. Case tools as organizational change: Investigating incremental and radical changes in systems development. MIS quarterly, pages 309–340, 1993.
[28]
A. Porter, H. Siy, A. Mockus, and L. Votta. Understanding the sources of variation in software inspections. ACM Transactions on Software Engineering and Methodology (TOSEM), 7(1):41–79, 1998.
[29]
P. C. Rigby, A. Bacchelli, G. Gousios, and M. Mukadam. A mixed methods approach to mining code review data: Examples and a study of multi-commit reviews and pull requests. In C. Bird, T. Menzies, and T. Zimmermann, editors, The Art and Science of Analyzing Software Data. Morgan Kaufmann, 2015.
[30]
P. C. Rigby and C. Bird. Convergent contemporary software peer review practices. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, pages 202–212. ACM, 2013.
[31]
P. C. Rigby, B. Cleary, F. Painchaud, M. Storey, and D. M. German. Contemporary peer review in action: Lessons from open source development. Software, IEEE, 29(6):56–61, 2012.
[32]
P. C. Rigby and M.-A. Storey. Understanding broadcast based peer review on open source software projects. In Proceedings of the 33rd International Conference on Software Engineering, pages 541–550. ACM, 2011.
[33]
E. M. Rogers. Diffusion of Innovations. Free Press, 5th edition, 2003.
[34]
M.-L. Sánchez-Gordón and R. V. O’Connor. Understanding the gap between software process practices and actual practice in very small companies. Software Quality Journal, pages 1–22, 2015.
[35]
C. Sauer, D. R. Jeffery, L. Land, and P. Yetton. The effectiveness of software development technical reviews: A behaviorally motivated program of research. Software Engineering, IEEE Transactions on, 26(1):1–14, 2000.
[36]
K. Spohrer, T. Kude, C. T. Schmidt, and A. Heinzl. Knowledge creation in information systems development teams: The role of pair programming and peer code review. In ECIS, page 213, 2013.
Information & Contributors
Information
Published In
FSE 2016: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering
November 2016
1156 pages
Copyright © 2016 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 November 2016
Permissions
Request permissions for this article.
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
Acceptance Rates
Overall Acceptance Rate 17 of 128 submissions, 13%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- Downloads (Last 12 months)76
- Downloads (Last 6 weeks)5
Reflects downloads up to 10 Feb 2025
Other Metrics
Citations
- Gao CHu XGao SXia XJin Z(2025)The Current Challenges of Software Engineering in the Era of Large Language ModelsACM Transactions on Software Engineering and Methodology10.1145/3712005Online publication date: 13-Jan-2025
- Davila NNunes IWiese I(2025)A fine-grained taxonomy of code review feedback in TypeScript projectsEmpirical Software Engineering10.1007/s10664-024-10604-y30:2Online publication date: 1-Mar-2025
- Sarma AChen N(2024)Effective Teaching through Code Reviews: Patterns and Anti-patternsProceedings of the ACM on Software Engineering10.1145/36607641:FSE(1262-1283)Online publication date: 12-Jul-2024
- Davila NMelegati JWiese I(2024)Tales From the Trenches: Expectations and Challenges From Practice for Code Review in the Generative AI EraIEEE Software10.1109/MS.2024.342843941:6(38-45)Online publication date: 1-Nov-2024
- Morikawa YKashiwa YFujiwara KIida H(2024)RevToken: A Token-Level Review Recommendation: How Far Are We?2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58944.2024.00068(654-659)Online publication date: 6-Oct-2024
- Papotti AParamitha RMassacci F(2024)On the acceptance by code reviewers of candidate security patches suggested by Automated Program Repair toolsEmpirical Software Engineering10.1007/s10664-024-10506-z29:5Online publication date: 3-Aug-2024
- Dorner MMendez DWnuk KZabardast ECzerwonka J(2024)The upper bound of information diffusion in code reviewEmpirical Software Engineering10.1007/s10664-024-10442-y30:1Online publication date: 17-Oct-2024
- Nanthaamornphong ABoonchieng E(2023)An Exploratory Study on Code Smells during Code Review in OSS Projects: A Case Study on OpenStack and WikiMediaRecent Advances in Computer Science and Communications10.2174/266625581666623022211231316:7Online publication date: Sep-2023
- Yang LXu JZhang YZhang HBacchelli AChandra SBlincoe KTonella P(2023)EvaCRC: Evaluating Code Review CommentsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616245(275-287)Online publication date: 30-Nov-2023
- Badampudi DUnterkalmsteiner MBritto R(2023)Modern Code Reviews—Survey of Literature and PracticeACM Transactions on Software Engineering and Methodology10.1145/358500432:4(1-61)Online publication date: 26-May-2023
- Show More Cited By
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.