Investigating Autonomous Agent Contributions in the Wild: Activity Patterns and Code Change over Time
Summary: arXiv:2604.00917v1 Announce Type: cross
Abstract: The rise of large language models for code has reshaped software development. Autonomous coding agents, able to create branches, open pull requests, and perform code reviews, now actively contribute to real-world projects. Their growing role offers a unique and timely opportunity to investigate AI-driven contributions and their effects on code quality, team dynamics, and software maintainability.
In this work, we construct a novel dataset of approximately 110,000 open-source pull requests, including associated commits, comments, reviews, issues, and file changes, collectively representing millions of lines of source code. We compare five popular coding agents, including OpenAI Codex, Claude Code, GitHub Copilot, Google Jules, and Devin, examining how their usage differs in various development aspects such as merge frequency, edited file types, and developer interaction signals, including comments and reviews.
Key Findings
- Agent Activity: Our findings indicate an increasing agent activity in open-source projects.
- Churn Rates: Contributions from AI agents are associated with more churn over time compared to human-authored code.
- Longitudinal Estimates: We provide several longitudinal estimates of survival and churn rates for agent-generated versus human-authored code.
Research Methodology
To explore the contributions of autonomous agents, we focused on a comprehensive dataset that includes:
- Pull Requests: Analyzed approximately 110,000 pull requests from various open-source projects.
- Code Reviews: Evaluated the interactions in comments and reviews to measure developer engagement.
- File Changes: Assessed the types of files that were edited and the specific changes made by both AI agents and human developers.
Implications for Software Development
The integration of autonomous agents into software development processes is not merely an enhancement but a significant shift. As coding agents become more prevalent, understanding their impact on:
- Code Quality: Assessing whether AI-generated code meets the quality standards set by human developers.
- Team Dynamics: Investigating how human teams interact with AI agents and the implications for collaboration.
- Software Maintainability: Analyzing the long-term effects of AI contributions on the maintainability of codebases.
Conclusion
As the role of autonomous agents in software development evolves, this research highlights the necessity of understanding their contributions in depth. While AI agents offer new capabilities, the higher churn rates associated with their contributions suggest that further research is required to ensure the sustainability of software projects. The insights gained from this study will be invaluable for developers, project managers, and researchers as they navigate the complexities of integrating AI into the software development lifecycle.