{"resourceId":"quailbot-boston-college-stm-agent-20260923","versions":[{"version":"external-0e5b94862e225418af94b5957e9118a968f99ebe6c5d4358fb4e0a63f154e21b","resource":{"id":"quailbot-boston-college-stm-agent-20260923","title":"Quailbot (Boston College): LLM agents complete STM tip conditioning with independent BraggZ verification; hidden-state failures persist","organization":"Boston College Department of Physics — arXiv preprint","sector":"Higher education physical sciences / autonomous instrumentation","geography":"United States (Chestnut Hill, MA); transferable methods with limits","publishedAt":"September 23, 2026 (arXiv)","publicationDate":"2026-09-23","eventDate":null,"sourceName":"arXiv 2609.27302","sourceLabel":"Academic preprint (arXiv)","sourceUrl":"https://arxiv.org/abs/2609.27302","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["developers-agents","operating-model","infrastructure"],"finding":"Cheng et al. introduce quailbot, an agent harness that places frontier LLMs inside a scanning tunneling microscope feedback loop for tip conditioning—a historically expert-intensive task with inaccessible tip-apex states. After brief human apprenticeship, launch-only agents autonomously completed end-to-end tip conditioning and passed independent human-expert verification (BraggZ delivered endpoints 79.5–295.4 vs initial ~20.7–41.6). Contingency experiments (bent tip, undisclosed piezo walker jam, mechanical sample rotation) show agents can recognize abnormal readbacks yet fail to identify unmodeled root causes or execute interventions unavailable through the software interface without human disclosure or physical takeover.","sledRelevance":"University AI-for-science programs wiring agents to real instruments need evidence of independent verification and explicit observability/action limits—not only successful launch narratives.","evidence":"arXiv abs 2609.27302 opened via HTTPS; launch-only autonomy claim, independent verification, BraggZ ranges, and contingency cases taken from opened abstract and results text (paper dated 23 Sep 2026).","architectureImplications":"Workspace-declared instrument capabilities with Get/Set/Ramp/Act CLI tools, linked observables after state-changing calls, experiment logs, and memory/skills modules; mutating tools gated by operator flag.","governanceImplications":"Human apprenticeship, safe gates, and independent post-run verification are required operating controls; agent self-report is not treated as success.","securityPrivacyImplications":"Mutating instrument tools default-denied; workspace limits which voltages/actions agents may invoke—instrument safety is a first-class constraint for campus deployments.","caveats":"Preprint; few runs on one STM/sample class; apprenticeship precedes autonomy; BraggZ aids but does not alone define delivery; spin-polarized tip was incidental not tasked.","streamIds":["research"],"roles":{"sales":"Sales framing: Research leaders buying instrument-agent platforms need measured end-to-end physical autonomy under independent verification, not chatbot demos. Quailbot’s STM tip-conditioning results (launch-only delivery; BraggZ lift; human-expert verification) support a bounded story about agents in real feedback loops. Do not sell guaranteed root-cause diagnosis of bent tips or jammed walkers—the paper’s contingency table shows where autonomy fails without human disclosure or physical intervention outside the software interface.","engineering":"Engineering framing: Fit is an LLM-in-the-loop harness with declared capabilities, linked readbacks, experiment logging, and mutating-tool gates around a real SPM controller. Integrate workspace schemas, safe gates, and independent scoring (for example BraggZ plus expert review) before claiming autonomy. Prerequisites include apprenticeship corpora, linked observables, and human takeover paths. Proof-of-value should be launch-only success under independent verification plus documented failure modes—not agent narrative alone.","delivery":"Delivery framing: Stand up workspace declarations, apprenticeship SOPs, independent verification metrics, and escalate-to-human playbooks before production instrument agents. Ownership: instrument owners control mutating capabilities; PIs own scientific claims; agents only act within declared limits. Acceptance requires launch-only success under independent verify and complete contingency logs. Risks include over-trusting completion reports, missing unmodeled states, and unsafe actions if capability gates are too broad."},"roleDetails":{"sales":{"customerProblem":"Expert-intensive instrument prep (e.g., STM tip conditioning) consumes scarce specialist time under uncertain action outcomes.","stakeholders":["Core facility / instrument directors","VPR / research computing","PI experimentalists","Research integrity / safety","Platform engineering"],"discoveryQuestions":["Which instrument actions are software-exposed vs require physical intervention?","Is success scored by independent metrics or agent self-report?","What apprenticeship and safe-gate policy exists?","How are contingency failures logged and escalated?"],"valueHypothesis":"An LLM instrument harness with linked readbacks and independent verification can complete some end-to-end conditioning tasks after apprenticeship when public states and actions suffice.","potentialEngagement":"Pilot on one instrument class with launch-only runs, independent scoring, and a pre-registered contingency escalation checklist.","unsupportedClaims":["Guaranteed bent-tip diagnosis","Zero human oversight forever","Transfer to every instrument class","Peer-reviewed materials outcomes from tip prep alone"]},"engineering":{"fit":"Agent harness wrapping SPM controller APIs with capability allowlists, linked observables, and experiment logs.","architectureIntegration":"Wire Get/Set/Ramp/Act tools, workspace JSON, context hooks for linked readbacks, and human mutate-enable flags; keep scoring outside agent self-report.","prerequisites":["API-exposed controller","Workspace capability map","Apprenticeship corpus","Independent image/FFT scoring","Human takeover path"],"constraints":["Hidden tip-apex state","Uncertain pulse outcomes","Preprint/few-run evidence","Actions not in software interface"],"security":"Default-deny mutating tools; restrict voltage/drive capabilities; retain immutable experiment logs for audit.","proofOfValue":["Launch-only delivery rate","Independent BraggZ/expert verify","Documented contingency handling","Time-to-ready tip vs human baseline"]},"delivery":{"work":"Implement harness, apprenticeship, verification pipeline, and escalation SOP; run launch-only pilots.","dependencies":["Instrument hardware uptime","Controller driver","Expert validators","Safety review"],"ownership":"Facility owns instrument safety gates; platform team owns harness; PIs own scientific interpretation.","skillsAdoption":"Train operators on workspace authoring and takeover; train PIs to read verification metrics not agent summaries.","governanceCheckpoints":["Capability allowlist review","Apprenticeship sign-off","Independent verify rubric freeze","Incident review after contingencies"],"acceptanceCriteria":["Launch-only success under independent verify","Contingency log complete","No unsafe mutate outside gates"],"risks":["Self-report as success","Unmodeled hardware faults","Over-broad action spaces","Single-instrument overfitting"]}},"retrievedAt":"2026-09-27T22:08:21-05:00","enrichedAt":"2026-09-27T22:13:16-05:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Reduces tip-conditioning specialist load only after apprenticeship investment; facility staff still needed for physical interventions and verification.","procurementImplications":"Require linked-observable logging, mutate gates, and independent verification in instrument-agent RFPs; fund expert validation time separately from model licenses.","operatingModelImplications":"Instrument autonomy is an operating model of apprenticeship + gated actions + independent verify, not unattended agents with completion chat.","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2609.27302","referenceExcerpt":"frontier LLM agents autonomously completed end-to-end tip conditioning on an STM and passed an independent verification","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}