From the Research Universities edition of October 10, 2026
Harvard study of 700+ firms: AI coding agents raise code volume 30% but not software output as review time rises 49%
Fiona Chen and James Stratton (Harvard University), reported by Ars Technica; data from Jellyfish engineering analytics · Software engineering productivity research (limited relevance to research software) · 700+ software firms (data 2021–March 2026)
- Publisher
- Ars Technica
- Original publication
- Ars Technica, Kyle Orland, 9 October 2026
- Source retrieved
- 2026-10-11
What happened
Using Jellyfish analytics (300 million work events, 700,000+ employees, 700+ firms, 2021–March 2026) and a difference-in-differences design, the researchers report AI coding agents raised lines of code 30%, commits 20% and pull requests 23%, but Jira issue/epic resolution did not change significantly. Review time from PR submission to merge rose 49%, the share of PRs with changes requested nearly doubled, comments per PR rose 35%, and the share of workers doing reviews rose 14%. AI agents produced 23.3% of review comments and 10.8% of PRs; no significant employment change attributed to AI.
Why it matters
Limited, transferable relevance: research software engineering groups and research computing teams adopting coding agents should expect review capacity, not code generation, to bound throughput. The firms are commercial, not universities.
Evidence and measured results
Opened the Ars Technica article via HTTPS (curl, full text extracted and read). The underlying Harvard paper was not opened; figures are as reported by Ars.
Limitations and uncertainty
Secondary reporting; primary paper not inspected; commercial firms only; data cutoff March 2026 predates newer agents; Jellyfish customers may not be representative; measures output proxies, not code quality or scientific correctness.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-11; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Research software teams adopting coding agents may see more code but no more delivered, reviewed software.
- Stakeholders
- Research software engineering leads
- Research computing directors
- Lab PIs maintaining shared code
- Discovery questions
- How are PR cycle time and issue resolution measured today?
- Who reviews agent-generated code?
- Is test automation sufficient to absorb more changes?
- Value hypothesis
- Investing in review and test capacity alongside agents may convert extra code into delivered features; unmeasured in research settings.
- Potential engagement
- Two-week baseline and review-capacity assessment before or during a coding-agent pilot.
- Unsupported claims
- Faster software delivery from agents
- Staff reductions
- Direct applicability to university research code
Pre-sales engineering
Role takeaway
- Fit
- Research software groups with Git-based workflows considering coding agents.
- Architecture and integration
- Repository analytics for PR cycle time, CI pipelines with automated tests, and review tooling (interpretation).
- Prerequisites
- Baseline repository metrics
- CI and test coverage
- Defined review policy
- Constraints
- Commercial-firm sample
- Data before newer agents
- Primary paper not inspected
- Security
- Require human review of agent-written changes touching data access, credentials or published analysis code.
- Proof of value
- Compare PR cycle time before and after pilot
- Track issue resolution rate, not commits
- Measure share of agent PRs needing changes
Delivery
Role takeaway
- Work
- Establish baseline metrics, define review rules for agent code, expand test automation, and monitor reviewer load.
- Dependencies
- Repository analytics access
- Agent licenses
- Reviewer availability
- Ownership
- Research software engineering lead with research computing management.
- Skills and adoption
- Train developers in reviewing agent output and writing tests that catch agent errors.
- Governance checkpoints
- Review policy approval
- Mid-pilot metrics review
- Decision gate before broader rollout
- Acceptance criteria
- Proposed: issue resolution rate does not fall during pilot
- Proposed: PR review time increase under 20%
- Proposed: 100% of agent PRs human-reviewed
- Risks
- Reviewer overload
- Unreviewed code in research pipelines
- Over-reading output metrics
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
CI, automated tests and review tooling become the throughput constraint when agents generate code.
Governance
Who approves, reviews and stays accountable for outcomes?
Code-review policies and accountability for agent-written code need explicit definition, especially for research code underpinning publications.
Security and privacy
What data, permissions and controls need testing?
Higher volumes of agent-generated code increase review load for security-relevant changes; not measured in the study.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Reviewer workload grows; teams need review skills and capacity planning rather than fewer staff.
Procurement
What should contracts, pricing and exit terms secure?
Agent licenses alone do not raise delivered output; budget for review and testing capacity.
Operating model
Which teams own the service once it runs?
Shifts effort from writing to reviewing code; research software teams should plan staffing accordingly.
Publication history
- 2026-10-10Research Universities · Issue 332 resources
Stable resource ID: harvard-ai-coding-agents-review-bottleneck-jellyfish-20261009