How Do So ware Developers Use ChatGPT? An Exploratory Study on GitHub Pull Requests

Moataz Chouchen, Narjes Bessghaier, Mahi Begoug, Ali Ouni, Eman Abdullah Alomar, Mohamed Wiem Mkaouer

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Nowadays, Large Language Models (LLMs) play a pivotal role in software engineering. Developers can use LLMs to address software development-related tasks such as documentation, code refactoring, debugging, and testing. ChatGPT, released by OpenAI, has become the most prominent LLM. In particular, ChatGPT is a cutting-edge tool for providing recommendations and solutions for developers in their pull requests (PRs). However, little is known about the characteristics of PRs that incorporate ChatGPT compared to those without it and what developers usually use it for. To this end, we quantitatively analyzed 243 PRs that listed at least one ChatGPT prompt against a representative sample of 384 PRs without any ChatGPT prompts. Our findings show that developers use ChatGPT in larger, time-consuming pull requests that are five times slower to be closed than PRs that do not use ChatGPT. Furthermore, we perform a qualitative analysis to build a taxonomy of the topics developers primarily address in their prompts. Our analysis results in a taxonomy comprising 8 topics and 32 sub-topics. Our findings highlight that ChatGPT is often used in review-intensive pull requests. Moreover, our taxonomy enriches our understanding of the developer's current applications of ChatGPT.CCS CONCEPTS• Software and its engineering → Collaboration in software development.

Original languageEnglish
Title of host publicationProceedings - 2024 IEEE/ACM 21st International Conference on Mining Software Repositories, MSR 2024
Pages212-216
Number of pages5
ISBN (Electronic)9798400705878
DOIs
StatePublished - 2024
Event21st IEEE/ACM International Conference on Mining Software Repositories, MSR 2024 - Lisbon, Portugal
Duration: 15 Apr 202416 Apr 2024

Publication series

NameProceedings - 2024 IEEE/ACM 21st International Conference on Mining Software Repositories, MSR 2024

Conference

Conference21st IEEE/ACM International Conference on Mining Software Repositories, MSR 2024
Country/TerritoryPortugal
CityLisbon
Period15/04/2416/04/24

Keywords

  • ChatGPT
  • Large Language Models
  • Manual analysis
  • Mining Software Repositories
  • Pull Requests

Fingerprint

Dive into the research topics of 'How Do So ware Developers Use ChatGPT? An Exploratory Study on GitHub Pull Requests'. Together they form a unique fingerprint.

Cite this