{"resourceId":"arxiv-poisoning-the-watchtower-log-substrate-prompt-injection-soc-copilots-preprint-2605-24421","versions":[{"version":"external-da6da4d6e01f89f8d4b9eabec50f638156d0bc9cf45e0e3a2e2aeae27971b727","resource":{"id":"arxiv-poisoning-the-watchtower-log-substrate-prompt-injection-soc-copilots-preprint-2605-24421","title":"Poisoning the Watchtower (preprint): summarization is the riskiest SOC copilot task, with log-based injections reaching 96% success undefended and 38% with constrained output","organization":"Rohan Pandey (DigitalOcean) and Archit Bhujang (Arizona State University)","sector":"Academic and industry security research / LLM-based security operations","geography":"United States (authors); not jurisdiction-specific","publishedAt":"arXiv preprint 2605.24421v1, submitted May 23, 2026","publicationDate":"2026-05-23","eventDate":null,"sourceName":"arXiv","sourceLabel":"Poisoning the Watchtower: Prompt Injection Attacks Against LLM-Augmented Security Operations Through Adversarial Log Content (arXiv:2605.24421)","sourceUrl":"https://arxiv.org/abs/2605.24421","evidenceClass":"academic-research","outcomeClass":"cautionary","topics":["developers-agents","data-security"],"finding":"Preprint, not peer reviewed. Using gpt-4o-mini as the analyst across 48 strategy, defense and task combinations, the authors found direct override instructions ineffective (0% suppression in classification), but persona hijacks suppressed 68% of malicious logs under a naive classifier. Summarization was the highest-risk task: context manipulation reached 96% injection success without defenses and 38% with constrained output. Average injection success fell from 26.6% with naive prompting to 11.8% under the strongest defense. A deterministic mock analyst mispredicted real model behavior. The authors conclude SOC copilots should treat raw log content as adversarial input.","sledRelevance":"Why now: independent corroboration, using a different model and taxonomy, of the risk that applies to SOC agents expanded in Palo Alto's Oct. 30 XSIAM release. Incident summaries are exactly what small public-sector teams lean on AI to write, and this study finds summarization the most exposed task. It does not test any Palo Alto product.","evidence":"Opened the arXiv abstract page and read the HTML full text, including the taxonomy, evaluation design, results, the mock-analyst comparison and the limitations section (one model, synthetic logs, no tool-using agents).","architectureImplications":"Constrained output formats reduce but do not stop injection; the authors note tool-using agents that query SIEM data or act likely present a larger attack surface than they tested.","governanceImplications":"Interpretation: incident summaries produced by AI should be labeled and reviewed before they feed reports to leadership or regulators.","securityPrivacyImplications":"Attacker-controlled fields such as user agents, URLs, DNS queries and attempted usernames can carry instructions to the model alongside intrusion evidence.","caveats":"Preprint, not peer reviewed. Single model (gpt-4o-mini) at one point in time; synthetic logs modeled on CIC-IDS2017 and UNSW-NB15, not production traffic; no multi-turn or tool-using agent tests. Commercial products may behave differently; transferability to SLED deployments must be tested.","streamIds":["palo-alto-networks"],"roles":{"sales":"Sales framing: this study reinforces a simple message for public-sector SOC leaders: AI-written incident summaries are the most manipulable output. The customer problem is relying on summaries to brief executives without checking them. A bounded engagement could add adversarial-log checks to an AI SOC evaluation. The study used one older model and synthetic logs, so it cannot support claims about any vendor product, and it should be presented as corroborating, not definitive.","engineering":"Engineering framing: the useful technical points are that direct overrides failed while persona and context manipulation succeeded, and that constrained output reduced summarization attacks from 96% to 38% without eliminating them. Engineers evaluating XSIAM agents or similar tools should test summarization with context-manipulation payloads in realistic log fields, compare summaries with raw evidence, and avoid relying on simulated or mock testing, which the authors found mispredicts real model behavior.","delivery":"Delivery framing: add an adversarial-summary check to the AI SOC acceptance plan and to periodic quality review. The SOC lead owns summary quality; communications and leadership briefings use only reviewed summaries. Proposed acceptance includes a documented summarization test and a review step before summaries leave the SOC. Risks include treating AI summaries as final, testing only classification, and assuming results from one model apply to the product actually deployed."},"roleDetails":{"sales":{"customerProblem":"Small public-sector SOCs rely on AI-written incident summaries that attacker-controlled log text can distort.","stakeholders":["SOC leads","CISOs who brief executives","Incident response coordinators"],"discoveryQuestions":["Are AI-generated summaries reviewed before reaching leadership?","Has summarization been tested with adversarial log content?","Which log fields reach the model unfiltered?"],"valueHypothesis":"Interpretation: a review step on AI summaries is a low-cost control against the highest-risk task identified here.","potentialEngagement":"Add adversarial-summarization tests to an existing AI SOC evaluation.","unsupportedClaims":["Behavior of any commercial product","Results for newer or larger models","Peer-reviewed status"]},"engineering":{"fit":"Relevant to SLED SOCs using AI summarization; corroborates rather than replaces product testing.","architectureIntegration":"Log fields from internet-facing services enter model context through SIEM queries and summaries.","prerequisites":["Representative log samples","Access to the product's summarization feature","Comparison against raw evidence"],"constraints":["Single-model synthetic study","No tool-using agents tested","Model behavior changes over time"],"security":"Treat raw logs as adversarial input; constrain outputs and review summaries.","proofOfValue":["Test context-manipulation payloads on summaries","Test persona-hijack payloads on classification","Do not rely on mock analysts for assurance"]},"delivery":{"work":"Add summarization-specific adversarial checks and a review step to AI SOC workflows.","dependencies":["Test corpus","Product access","Incident workflow change"],"ownership":"The SOC lead owns summary quality and review.","skillsAdoption":"Analysts need practice comparing AI summaries against raw logs.","governanceCheckpoints":["Summarization test before go-live","Review before external or executive use"],"acceptanceCriteria":["Proposed: documented adversarial-summary test per deployed model","Proposed: 100% of AI summaries reviewed before executive briefing in pilot"],"risks":["AI summaries treated as final","Classification tested but not summarization","Single-model results over-generalized"]}},"retrievedAt":"2026-11-01T03:07:01Z","enrichedAt":"2026-11-01T03:12:18Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Reinforces the need for analysts who can check AI summaries against raw evidence.","procurementImplications":"Interpretation: evaluation criteria for AI SOC tools can include summarization behavior under adversarial logs.","operatingModelImplications":"Use AI summaries as drafts; keep raw-evidence review in the incident workflow.","peerReview":"preprint","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2605.24421","referenceExcerpt":"These results suggest that SOC copilots should treat raw log content as adversarial input rather than ordinary analyst context.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}