Papers
arxiv:2509.14745

On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub

Published on Sep 18
ยท Submitted by Leo on Sep 25
Authors:
,
Hao Li ,
,
,
,

Abstract

Agent-assisted pull requests generated by Claude Code are largely accepted in open-source projects, with most requiring minimal human modification.

AI-generated summary

Large language models (LLMs) are increasingly being integrated into software development processes. The ability to generate code and submit pull requests with minimal human intervention, through the use of autonomous AI agents, is poised to become a standard practice. However, little is known about the practical usefulness of these pull requests and the extent to which their contributions are accepted in real-world projects. In this paper, we empirically study 567 GitHub pull requests (PRs) generated using Claude Code, an agentic coding tool, across 157 diverse open-source projects. Our analysis reveals that developers tend to rely on agents for tasks such as refactoring, documentation, and testing. The results indicate that 83.8% of these agent-assisted PRs are eventually accepted and merged by project maintainers, with 54.9% of the merged PRs are integrated without further modification. The remaining 45.1% require additional changes benefit from human revisions, especially for bug fixes, documentation, and adherence to project-specific standards. These findings suggest that while agent-assisted PRs are largely acceptable, they still benefit from human oversight and refinement.

Community

Paper author Paper submitter

๐Ÿš€ ๐—”๐—ด๐—ฒ๐—ป๐˜๐—ถ๐—ฐ ๐—ฃ๐—ฅ๐˜€ ๐—ฎ๐—ฟ๐—ฒ ๐˜€๐—ต๐—ถ๐—ฝ๐—ฝ๐—ถ๐—ป๐—ด โ€“ ๐Ÿด๐Ÿฏ.๐Ÿด% ๐—บ๐—ฒ๐—ฟ๐—ด๐—ฒ ๐—ฟ๐—ฎ๐˜๐—ฒ ๐Ÿš€

Not a demo, not a toy. We study Claude Code PRs on GitHub, agentic PRs are merged 83.8% of the time vs 91.0% for humans, with similar median merge speeds (1.23 hrs vs 1.04 hrs).

โ€ข ๐—ง๐—ต๐—ฒ ๐˜€๐˜๐—ผ๐—ฟ๐˜† ๐—ผ๐—ป ๐˜๐—ต๐—ฒ ๐—ด๐—ฟ๐—ผ๐˜‚๐—ป๐—ฑ: agents accelerate setup and routine improvements; humans carry context, enforce quality, and keep scope tight.
โ€ข ๐—ช๐—ต๐—ฎ๐˜ ๐—ฎ๐—ด๐—ฒ๐—ป๐˜๐˜€ ๐—ฑ๐—ผ ๐—บ๐—ผ๐—ฟ๐—ฒ: refactoring, tests, and docs.
โ€ข ๐—ช๐—ต๐˜† ๐—ฟ๐—ฒ๐—ท๐—ฒ๐—ฐ๐˜๐—ถ๐—ผ๐—ป๐˜€ ๐—ต๐—ฎ๐—ฝ๐—ฝ๐—ฒ๐—ป: alternative solutions, oversized PRs, or obsolescence โ€“ not simply โ€œbad AI codeโ€.
โ€ข ๐—ช๐—ต๐—ฎ๐˜ ๐—ฟ๐—ฒ๐˜ƒ๐—ถ๐—ฒ๐˜„๐—ฒ๐—ฟ๐˜€ ๐˜€๐˜๐—ถ๐—น๐—น ๐—ณ๐—ถ๐˜…: bugs (45.1%), docs (27.4%), refactoring (25.7%), style (22.1%) before merge.
โ€ข ๐—ช๐—ต๐—ฒ๐—ฟ๐—ฒ ๐˜๐—ต๐—ฒ๐˜† ๐˜€๐˜๐—ถ๐—น๐—น ๐˜€๐˜๐˜‚๐—บ๐—ฏ๐—น๐—ฒ: legacy-heavy codebases or cross-cutting PRs.

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2509.14745 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2509.14745 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2509.14745 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.