{"resourceId":"arxiv-cachelint-prompt-caching-mis-integration-llm-agents-bedrock-20261026","versions":[{"version":"external-46b8a803c60d516aa4ca6cdd1f4d0db90ddcba76b935b1c4abf79657cbc8f309","resource":{"id":"arxiv-cachelint-prompt-caching-mis-integration-llm-agents-bedrock-20261026","title":"Preprint: prompt-caching mis-integrations in LLM agents mostly fail silently; a Bedrock cache-lifetime change shows how unchanged code can become defective","organization":"Ying Zhang, Ruoqi Guo, Yi Liu, Gelei Deng, Yuekang Li, Yepang Liu, Leo Zhang (authors)","sector":"Academic software engineering research; LLM agent cost control","geography":"International (open-source agent projects)","publishedAt":"2026-10-26","publicationDate":"2026-10-26","eventDate":null,"sourceName":"arXiv","sourceLabel":"Silent and Costly: Characterizing and Detecting Prompt-Caching Mis-Integration in LLM Agents (arXiv:2610.34817v2)","sourceUrl":"https://arxiv.org/abs/2610.34817","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["developers-agents","infrastructure"],"finding":"Preprint, not peer reviewed (the authors state acceptance at ICSE 2027; no journal reference confirms publication). The authors studied 120 prompt-caching pull requests across 10 popular open-source agent systems spanning Anthropic, OpenAI, Gemini, Bedrock and OpenRouter-style gateways, and found mis-integrations on both the request side and in usage counting, most of which fail silently. They note Bedrock, like Anthropic, requires explicit opt-in and limits cache entries per request, and that Bedrock's opt-in one-hour cache lifetime turned previously correct code into a defect without any code change. One case left Bedrock-routed requests without cache markers. Their CacheLint tool reached 0.71 precision and 0.78 recall on the labelled set and found 32 mis-integrations across 571 projects at 0.86 precision, with 18 upstream fixes.","sledRelevance":"Agencies and universities paying per token for Bedrock-based agents, including tools like the GovCloud configurations in this edition, can overpay without any visible error if caching is misconfigured. The finding is independent and directly names Bedrock behaviour, which makes it useful for cost governance when SLED teams build or buy agents on AWS.","evidence":"Empirical study of open-source repositories plus a static analysis tool with precision and recall reported; real-world detection on 571 projects. Not a study of public-sector systems.","architectureImplications":"Caching correctness depends on provider-specific directives and time-varying capabilities; gateways that route to Bedrock can drop cache markers.","governanceImplications":"Cost controls should include periodic checks of cache hit rates and provider capability changes, since provider updates can silently change cost behaviour.","securityPrivacyImplications":"Not a security paper; misconfiguration is a cost and latency issue.","caveats":"Preprint; sample is open-source agent projects, not SLED systems; Bedrock is one of several providers studied, not the sole subject. Cost impact in dollars is not quantified in the abstract or the passages reviewed.","streamIds":["aws"],"roles":{"sales":"Bring this to clients as a cost-governance point, backed by independent research. Agencies paying per token for Bedrock agents may lose expected savings from prompt caching without any error appearing. The study names Bedrock behaviour directly, including a cache-lifetime change that broke previously correct code. A light FinOps review of AI workloads is a credible, bounded offer. Do not quote savings figures; the paper reports detection accuracy, not dollars.","engineering":"Engineering should check every Bedrock integration for explicit cache directives, cache-entry limits and the configured cache lifetime per model. Gateways and SDK wrappers deserve particular attention, since the authors found routed requests losing cache markers. Track cache read and write token metrics in CloudWatch and alert on drops. Re-test after provider announcements, because capability changes can alter behaviour without code changes.","delivery":"Delivery should add caching checks to AI workload acceptance and run them again after provider updates. Assign a FinOps owner to review cache metrics monthly and record provider capability changes that affect cost. Proposed acceptance criteria are documented cache configuration for every production model and a measurable cache-hit baseline. The study did not measure public-sector systems, so the client's own metrics establish the baseline."},"roleDetails":{"sales":{"customerProblem":"Teams running metered LLM agents may be paying more than expected because caching is silently misconfigured.","stakeholders":["Agency or campus FinOps lead","AI platform owner","Application developers"],"discoveryQuestions":["Do you track cache read and write tokens per model?","Do your agents route Bedrock calls through gateways or wrappers?","Who reviews provider capability changes?"],"valueHypothesis":"Checking caching configuration may recover expected cost and latency benefits; size of benefit is unknown until measured.","potentialEngagement":"Short AI workload FinOps review covering caching configuration and metrics.","unsupportedClaims":["Specific dollar savings","That SLED systems have these defects"]},"engineering":{"fit":"Any team building agents on Bedrock with prompt caching.","architectureIntegration":"Audit request directives, gateway routing and cache lifetime settings; monitor cache metrics in CloudWatch.","prerequisites":["Inventory of models and integration code","Access to token usage metrics"],"constraints":["Provider capabilities change over time","Study sample is open-source, not public sector"],"security":"Not a security issue; ensure monitoring does not log sensitive prompt content.","proofOfValue":["Measure cache hit rate before and after fixes","Run static checks on integration code","Re-test after provider updates"]},"delivery":{"work":"Add caching configuration review and monitoring to AI operations.","dependencies":["Metric availability","Developer time for fixes"],"ownership":"AI platform owner with FinOps lead.","skillsAdoption":"Developers learn provider-specific caching rules.","governanceCheckpoints":["Pre-production caching review","Monthly cost and cache metric review","Post-provider-change retest"],"acceptanceCriteria":["Proposed: documented cache configuration for each production model","Proposed: cache-hit baseline recorded and monitored"],"risks":["Silent regressions after provider changes","Over-optimising cost at the expense of answer freshness"]}},"retrievedAt":"2026-11-01T03:11:20Z","enrichedAt":"2026-11-01T03:16:34Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Developers need awareness of provider-specific caching rules.","procurementImplications":"Metered token contracts should include visibility into cache usage and billing detail to detect silent overspend.","operatingModelImplications":"Operations teams should monitor cache read and write metrics per model as part of FinOps for AI.","peerReview":"preprint","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2610.34817","referenceExcerpt":"More importantly, most mis-integrations fail silently without causing observable runtime failures.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}