From the AWS edition of October 31, 2026
Preprint: prompt-caching mis-integrations in LLM agents mostly fail silently; a Bedrock cache-lifetime change shows how unchanged code can become defective
Ying Zhang, Ruoqi Guo, Yi Liu, Gelei Deng, Yuekang Li, Yepang Liu, Leo Zhang (authors) · Academic software engineering research; LLM agent cost control · International (open-source agent projects)
- Publisher
- arXiv
- Original publication
- 2026-10-26
- Source retrieved
- 2026-11-01
What happened
Preprint, not peer reviewed (the authors state acceptance at ICSE 2027; no journal reference confirms publication). The authors studied 120 prompt-caching pull requests across 10 popular open-source agent systems spanning Anthropic, OpenAI, Gemini, Bedrock and OpenRouter-style gateways, and found mis-integrations on both the request side and in usage counting, most of which fail silently. They note Bedrock, like Anthropic, requires explicit opt-in and limits cache entries per request, and that Bedrock's opt-in one-hour cache lifetime turned previously correct code into a defect without any code change. One case left Bedrock-routed requests without cache markers. Their CacheLint tool reached 0.71 precision and 0.78 recall on the labelled set and found 32 mis-integrations across 571 projects at 0.86 precision, with 18 upstream fixes.
Why it matters
Agencies and universities paying per token for Bedrock-based agents, including tools like the GovCloud configurations in this edition, can overpay without any visible error if caching is misconfigured. The finding is independent and directly names Bedrock behaviour, which makes it useful for cost governance when SLED teams build or buy agents on AWS.
Evidence and measured results
Empirical study of open-source repositories plus a static analysis tool with precision and recall reported; real-world detection on 571 projects. Not a study of public-sector systems.
Limitations and uncertainty
Preprint; sample is open-source agent projects, not SLED systems; Bedrock is one of several providers studied, not the sole subject. Cost impact in dollars is not quantified in the abstract or the passages reviewed.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-01; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Teams running metered LLM agents may be paying more than expected because caching is silently misconfigured.
- Stakeholders
- Agency or campus FinOps lead
- AI platform owner
- Application developers
- Discovery questions
- Do you track cache read and write tokens per model?
- Do your agents route Bedrock calls through gateways or wrappers?
- Who reviews provider capability changes?
- Value hypothesis
- Checking caching configuration may recover expected cost and latency benefits; size of benefit is unknown until measured.
- Potential engagement
- Short AI workload FinOps review covering caching configuration and metrics.
- Unsupported claims
- Specific dollar savings
- That SLED systems have these defects
Pre-sales engineering
Role takeaway
- Fit
- Any team building agents on Bedrock with prompt caching.
- Architecture and integration
- Audit request directives, gateway routing and cache lifetime settings; monitor cache metrics in CloudWatch.
- Prerequisites
- Inventory of models and integration code
- Access to token usage metrics
- Constraints
- Provider capabilities change over time
- Study sample is open-source, not public sector
- Security
- Not a security issue; ensure monitoring does not log sensitive prompt content.
- Proof of value
- Measure cache hit rate before and after fixes
- Run static checks on integration code
- Re-test after provider updates
Delivery
Role takeaway
- Work
- Add caching configuration review and monitoring to AI operations.
- Dependencies
- Metric availability
- Developer time for fixes
- Ownership
- AI platform owner with FinOps lead.
- Skills and adoption
- Developers learn provider-specific caching rules.
- Governance checkpoints
- Pre-production caching review
- Monthly cost and cache metric review
- Post-provider-change retest
- Acceptance criteria
- Proposed: documented cache configuration for each production model
- Proposed: cache-hit baseline recorded and monitored
- Risks
- Silent regressions after provider changes
- Over-optimising cost at the expense of answer freshness
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Caching correctness depends on provider-specific directives and time-varying capabilities; gateways that route to Bedrock can drop cache markers.
Governance
Who approves, reviews and stays accountable for outcomes?
Cost controls should include periodic checks of cache hit rates and provider capability changes, since provider updates can silently change cost behaviour.
Security and privacy
What data, permissions and controls need testing?
Not a security paper; misconfiguration is a cost and latency issue.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Developers need awareness of provider-specific caching rules.
Procurement
What should contracts, pricing and exit terms secure?
Metered token contracts should include visibility into cache usage and billing detail to detect silent overspend.
Operating model
Which teams own the service once it runs?
Operations teams should monitor cache read and write metrics per model as part of FinOps for AI.
Publication history
- 2026-10-31AWS · Issue 037 resources
Stable resource ID: arxiv-cachelint-prompt-caching-mis-integration-llm-agents-bedrock-20261026