From the NVIDIA edition of October 25, 2026
Independent test (dev.to, Oct 23): NVIDIA OpenShell 0.1.5's OCSF audit export wrote zero records on a CI runner; author builds an external evidence layer
Independent developer (dev.to author techdevfun); NVIDIA OpenShell · AI agent security and audit evidence (cross-sector, applicable to public agencies) · Not location-specific (GitHub Actions runner)
- Publisher
- DEV Community (dev.to)
- Original publication
- 2026-10-23 (dev.to)
- Source retrieved
- 2026-10-26
- Event date
- 2026-10-23
What happened
The author cites an independent evaluation of NVIDIA OpenShell that found blocking held up but evidence did not: a blocked DNS query left no event, the OCSF audit export silently wrote nothing, and some events existed only on the VM console. The author then booted OpenShell 0.1.5 with the docker driver on a GitHub Actions runner and reports that the documented OCSF-JSON export produced zero records in both runs with ocsf_json_enabled set as documented, and that the supervisor file sink sits on a tmpfs external tools cannot read. Their own external layer (Agent Seatbelt, OPA/Rego policies plus a hash-chained ledger) reported 11/11 (clean) and 13/13 (planted) coverage, five planted violations blocked and recorded, no false blocks, and p95 decision latency of 27.5 ms. NVIDIA's OpenShell documentation separately says OCSF delivery is best-effort and retention depends on the runtime and external collector.
Why it matters
Why now: Onetera's Oct 25 city platform cites OpenShell for 'full audit logging'. For agencies subject to public-records, audit and due-process requirements, this test shows audit evidence from OpenShell must be verified in the actual deployment configuration.
Evidence and measured results
Single-author hands-on test with published method (version, driver, runner) and two runs, building on a cited prior evaluation; NVIDIA documentation (opened) confirms best-effort delivery. Small sample, one configuration, not peer reviewed; the author also promotes their own tool.
Limitations and uncertainty
One configuration (0.1.5, Docker driver, CI runner); results may differ on other drivers or later versions; author has a commercial interest in the add-on layer.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-26; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Agencies need defensible records of what AI agents did with resident data, but sandbox audit exports may be incomplete.
- Stakeholders
- CISO / security officer
- Records manager
- Internal auditor
- Agent platform vendor
- Discovery questions
- Where do agent audit events land today, and who reviews them?
- Has anyone tested that every agent action is logged?
- What retention period applies to agent logs?
- Value hypothesis
- Interpretation: verified audit evidence may reduce audit findings and make agent pilots easier to approve.
- Potential engagement
- Audit-evidence verification test for an agent pilot before go-live.
- Unsupported claims
- That OpenShell always fails to export (one configuration tested)
- That the author's add-on works in other environments
Pre-sales engineering
Role takeaway
- Fit
- Any OpenShell-based agent deployment handling records or consequential decisions.
- Architecture and integration
- OpenShell sandbox events forwarded to an external collector or SIEM, with scripted completeness checks.
- Prerequisites
- Agency log platform
- Test harness of scripted agent actions
- Constraints
- Single test configuration
- Best-effort delivery per NVIDIA docs
- Security
- Durable, tamper-evident external log storage; alert on missing events.
- Proof of value
- Scripted run with expected event count compared with collected records
- Repeat after upgrade
Delivery
Role takeaway
- Work
- Set up external log forwarding, completeness tests and retention for agent events.
- Dependencies
- Security operations
- Records management
- Platform vendor
- Ownership
- Agency security team owns log completeness; records manager owns retention.
- Skills and adoption
- Security staff learn agent event schemas (OCSF) and testing.
- Governance checkpoints
- Audit-completeness test before go-live
- Retest after each platform upgrade
- Acceptance criteria
- Proposed: 100% of scripted agent actions recorded externally
- Proposed: alert fires within a defined time when events are missing
- Risks
- Silent log loss
- Version drift
- Unretained evidence for appeals
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Treat the sandbox's own export as one input; ship events to an external, durable collector and verify completeness with test actions.
Governance
Who approves, reviews and stays accountable for outcomes?
Claims of 'full audit logging' in procurement should be contract terms with a verification test, not marketing statements.
Security and privacy
What data, permissions and controls need testing?
Missing audit records undermine incident investigation and records-retention compliance for agent actions on resident data.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Security and records staff need skills to test agent logging, not just review dashboards.
Procurement
What should contracts, pricing and exit terms secure?
Add acceptance tests requiring that every agent action in a scripted run appears in the agency's log system.
Operating model
Which teams own the service once it runs?
Audit evidence becomes an operated service with monitoring for gaps, owned by the agency or its provider.
Publication history
- 2026-10-25NVIDIA · Issue 508 resources
Stable resource ID: devto-openshell-ocsf-audit-export-zero-records-agent-seatbelt-20261023