{"resourceId":"auto-research-heasarc-ixpe-provenance-pipeline-20261103","versions":[{"version":"external-2c4ce84ba1df08c7421f0e5e403639f09bec2da1e6740df819715f867e0f42d1","resource":{"id":"auto-research-heasarc-ixpe-provenance-pipeline-20261103","title":"A semi-autonomous multi-agent pipeline for high-energy astrophysics research with enforced provenance and verifiable self-extension","organization":"Washington University in St. Louis (Physics Department)","sector":"University research (high-energy astrophysics data analysis; AI research agents)","geography":"United States (WashU; HEASARC/IXPE public data; transferable caution for any campus deploying analysis agents on shared archives)","publishedAt":"3 November 2026 (arXiv 2611.05251v1)","publicationDate":"2026-11-03","eventDate":null,"sourceName":"arXiv","sourceLabel":"Zhou, F., \"A semi-autonomous multi-agent pipeline for high-energy astrophysics research with enforced provenance and verifiable self-extension,\" arXiv:2611.05251, 3 Nov 2026","sourceUrl":"https://arxiv.org/abs/2611.05251","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["developers-agents","knowledge-work","operating-model"],"finding":"Fang Zhou (Washington University in St. Louis Physics) presents auto_research, a semi-autonomous multi-agent pipeline that retrieves raw HEASARC data, writes analysis tools it needs, and drafts reports. Tools enter the toolkit only after automatic checks (repeated test runs and a scan for phrases that accompany invented values) plus code review by a model from a different provider; a human decides whether to promote a tool to the trusted branch. A number in a report passes only if recorded toolkit output or the request contains it, or if the report gives its location in a cited paper. On IXPE data the pipeline reproduces published Cyg X-1 results: all 14 compared quantities agree within 3 sigma, though polarization degrees are lower than the published values. With weaker models in the main roles it recovers 12 of 14 quantities and reports the other two as not measured. Planted-defect tests are cautionary: of 65 defects planted in the toolkit, 47 pass the automatic checks and one also passes the code review; the review also rejects 40 of 69 correct changes. The pipeline was also used for polarization-stability analysis of six observations, a hardness survey of 29 observations, and a Stokes q–u lag search; the author notes it did not recognize that uncorrected absorption distorted a combined PD–hardness correlation. Included now because this astro-ph.IM preprint (3 Nov) was not in the 4–6 Nov editions and supplies measured provenance-gate results complementary to this week’s agent-evaluation cluster.","sledRelevance":"Directly relevant to research universities that want analysis agents on shared observational archives: shows a workable provenance contract for reported numbers and a human promotion step, while quantifying how often automatic checks and cross-provider code review still miss planted defects.","evidence":"Single-author arXiv preprint (astro-ph.IM/HE) with a published-result reproduction (Cyg X-1, 14 quantities), a planted-defect/control benchmark (65 defects and 69 defect-free changes), a weaker-model ablation, and additional IXPE analyses. Not peer-reviewed.","architectureImplications":"Separate untrusted tool-writing from a trusted toolkit branch; require deterministic test reruns, fabrication-phrase scans, cross-provider review, and a validator that admits report numbers only when they appear in recorded tool output, the request, or a cited-paper locator.","governanceImplications":"Keep a human in the promotion path for new analysis tools. Treat automatic gates as advisory: most planted defects still passed the automatic checks, so research-integrity policy should not equate a green gate with verified science.","securityPrivacyImplications":"The pipeline retrieves public HEASARC/IXPE data and executes agent-written code. Campus deployments would still need sandboxing, secret hygiene, and controls for any proprietary or embargoed observations (the paper notes two requested observations were proprietary/encrypted and reported as not measured). The preprint does not specify campus SOC controls.","caveats":"Preprint; single author; IXPE/Cyg X-1 and related BHXRB analyses are the testbed, not a cross-discipline benchmark. Automatic checks missed 47/65 planted defects; review is conservative (rejects 40/69 correct changes). The pipeline does not propose its own scientific ideas and missed an absorption-correction issue in a combined correlation. 3 Nov date is slightly before the preferred 6–7 Nov window; selected as newly relevant and previously uncovered.","streamIds":["research"],"roles":{"sales":"Evaluation framing: a WashU Physics pipeline shows research agents can retrieve public HEASARC/IXPE data, write tools, and reproduce published Cyg X-1 results—all 14 compared quantities within 3 sigma—when reported numbers must trace to recorded toolkit output, the request, or a cited-paper location, and a human still promotes tools to the trusted branch. Use this with VPR and research-computing leads who want agent-assisted analysis without trusting unverified numbers. Pitch a provenance-gated pilot, not an autonomous astrophysics service: 47 of 65 planted defects still passed automatic checks, and the code review rejected 40 of 69 correct changes.","engineering":"Architecture framing: split the system into (1) agents that write candidate tools, (2) deterministic gates (rerun tests, fabrication-phrase scan, structural and provenance checks), (3) a cross-provider model reviewer, and (4) a human-promoted trusted branch whose recorded outputs are the only numbers a report validator will admit. Integrate with archive retrieval (here HEASARC) and hash-checked figure and output files. Do not treat the gate as sufficient: most planted defects passed the automatic checks, so keep conservative review and human promotion. Weaker models recovering 12 of 14 quantities and declaring two unmeasured is a useful honest-failure behavior to preserve.","delivery":"Delivery framing: stand up a scoped reproduction of a published analysis (the paper’s Cyg X-1 comparison is the template) before allowing agents to write new science tools. Owners are typically the PI plus research computing; a designated scientist promotes tools. Acceptance should require provenance-valid reports, planted-defect or mutation tests of the gate, and documented human promotion—not matching published numbers alone. Watch for scientific mistakes the pipeline does not catch, such as the uncorrected-absorption distortion the author reports in a combined PD–hardness correlation."},"roleDetails":{"sales":{"customerProblem":"Campus analysis agents can fabricate numbers or report values that did not come from the code they ran, undermining trust in agent-assisted astronomy or other archive science.","stakeholders":["Vice President for Research","Research computing director","Astrophysics or observational-science PIs","Research integrity / library data services"],"discoveryQuestions":["Do agent-written analysis reports currently require every number to appear in a recorded tool output?","Who promotes new agent-written tools into a trusted codebase?","Have you tested gates with planted defects, not only happy-path reproductions?","What happens when requested observations are proprietary or missing?"],"valueHypothesis":"Provenance validation plus human tool promotion may let agents reproduce published archive analyses (here 14/14 Cyg X-1 quantities within 3 sigma) while making fabricated numbers harder to publish—without implying the automatic gate is sufficient.","potentialEngagement":"A bounded HEASARC/IXPE-style reproduction pilot that instruments number-level provenance and measures planted-defect catch rates before any production science use.","unsupportedClaims":["Autonomous discovery of new astrophysics without human scientific judgment","That automatic checks catch most defective agent-written tools (47 of 65 planted defects passed them)","Transfer of Cyg X-1 reproduction rates to other instruments or disciplines without local tests"]},"engineering":{"fit":"Strong for computational analyses of public archives with reusable tool libraries; weaker for agents expected to invent research questions or for closed, regulated datasets.","architectureIntegration":"Archive retrieval to multi-agent analysis to coder-written candidate tool to deterministic gate plus cross-provider review to human promotion to trusted branch to a report validator that admits only provenance-backed numbers.","prerequisites":["Access to the relevant archive (HEASARC in the paper)","A trusted toolkit branch with well-defined return values","Compute for repeated tool tests and analysis runs","A designated human promoter"],"constraints":["Automatic checks missed 47 of 65 planted defects","Cross-provider review rejected 40 of 69 correct changes","Pipeline does not develop its own scientific ideas","Single-field IXPE/BHXRB testbed"],"security":"Sandbox agent-written code; treat retrieved public data separately from any embargoed observations; pin output hashes the validator checks.","proofOfValue":["Reproduction of a published analysis within a pre-agreed tolerance (paper: 14/14 Cyg X-1 quantities within 3 sigma)","Planted-defect catch rate for automatic checks vs review","Honest not-measured behavior when data are missing (paper: 2 of 14 with weaker models; 2 proprietary observations)"]},"delivery":{"work":"Define the trusted-toolkit interface; wire provenance validation; run a published-result reproduction; execute planted-defect tests; document human promotion.","dependencies":["PI scientific ownership","Research-computing execution environment","Archive credentials/access for public data"],"ownership":"PI owns scientific acceptance; research computing owns runtime and sandboxing; a named scientist promotes tools.","skillsAdoption":"Analysts shift from trusting agent prose to inspecting recorded outputs, hashes, and promotion diffs.","governanceCheckpoints":["No trusted-branch merge without human promotion","No report number without validator provenance","Periodic planted-defect retests after model or prompt changes"],"acceptanceCriteria":["Proposed: published-result quantities meet the agreed tolerance","Proposed: provenance failures block report completion","Proposed: planted-defect report retained even if catch rate is imperfect"],"risks":["False confidence from a green automatic gate","Reviewer over-rejection slowing tool growth","Scientific errors (for example uncorrected absorption) that pass technical provenance checks"]}},"retrievedAt":"2026-11-07T22:08:52-06:00","enrichedAt":"2026-11-07T22:14:26-06:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"May reduce repeated recoding of archive analyses for students and staff, while adding promotion-review workload and requiring skills to inspect recorded outputs rather than agent narrative.","procurementImplications":"When buying research-agent platforms, require number-level provenance, cross-checks by a separate model or tool, human promotion of new code, and planted-defect or mutation tests—not a demo reproduction alone.","operatingModelImplications":"Supports a shared-service pattern in which research computing hosts the gated runtime and the PI retains scientific promotion and interpretation rights.","peerReview":"preprint","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2611.05251","referenceExcerpt":"Of 65 defects that we planted in the toolkit, 47 pass the automatic checks, and one passes the code review as well","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}