From the Research Universities edition of November 9, 2026
More with Less: Efficient Parallel Scientific Code Translation via an Agent-Skill Workflow
South China University of Technology and Michigan Technological University · Research computing and HPC software (scientific code porting) · China and United States (authors' institutions); benchmark kernels, not site-specific deployments
- Publisher
- arXiv
- Original publication
- 6 November 2026 (arXiv 2611.08947v1, cs.DC; listed 9 November)
- Source retrieved
- 2026-11-10
What happened
The authors translate 30 HeCBench GPU kernels among CUDA, OpenMP and SYCL (six directions, three open-source LLMs) with three agents coordinated by a verdict-driven orchestrator that routes each failure to a targeted repair skill, and compare it with the LASSI baseline run on the same models. Their framework averages 88.7% verified correctness against 65.5% for LASSI using DeepSeek V4 Flash, higher in every one of 18 model and direction combinations, with 26.2% fewer tokens on average (1,071k vs 1,451k per direction) and up to 52.4% fewer overall. Results vary: SYCL as target is harder (74.6% vs 40.0% for LASSI), OpenMP to SYCL scores 79.2% vs 33.3%, and per-model averages span 78.6% to 94.6%. A stricter offline LLM verification lowers the framework's pass rates by 8.3 to 17.3 percentage points and identified 117 false positives that passed compiler and runtime checks, led by placeholder implementations (31), numerical errors (28) and skipped verification (25). Included now because it was submitted on 6 Nov, appeared in the 9 Nov arXiv listings and was not archived; it is research-computing evidence that automated pass signals overstate correctness when agents port scientific code.
Why it matters
Relevant to university research-computing and HPC centers facing CUDA-to-portable-model migrations: it shows what an agent translation loop can reach on kernels, how much a second verification stage changes the answer, and that scientific correctness needs checks beyond compilation success.
Evidence and measured results
arXiv preprint with a controlled comparison on 30 HeCBench kernels, three open-source models, six translation directions, a re-run LASSI baseline on identical models and environment, repair-round curves, a pass-rate comparison under two verification stages and a taxonomy of 117 false positives. Artifact-evaluation appendix is provided. Not peer reviewed.
Limitations and uncertainty
Preprint; kernel-level benchmarks only (30 kernels, six lack OpenMP versions so 24 per OpenMP-source direction), not multi-file applications; baseline is one prior system (LASSI) re-run with different models than its original study; the complementary verifier is itself an LLM and can err; correctness is numerical, with no performance results; low-resource SYCL targets remain weak; no cost or staffing data.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-09; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Research groups must move GPU codes between CUDA, OpenMP and SYCL as hardware changes, and manual porting is slow and error-prone.
- Stakeholders
- Research computing director
- HPC applications specialists
- Principal investigators with legacy GPU codes
- Research software engineers
- Discovery questions
- Which codes must be ported in the next 12 months and between which models?
- How is correctness of a ported code checked today?
- Who owns sign-off on numerical equivalence?
- Which target hardware and compilers are in use?
- Value hypothesis
- An agent translation loop with an independent verification stage may reduce first-draft porting effort on kernel-sized code while exposing silent failures, if reviewed by code owners.
- Potential engagement
- A bounded pilot porting and verifying a small set of kernels from two or three local codes, with a measured comparison of accepted ports and review effort.
- Unsupported claims
- That agents can port full applications
- Guaranteed time or cost savings
- That the LLM verifier removes the need for human review
- Performance parity of translated code
Pre-sales engineering
Role takeaway
- Fit
- Good for kernel-sized CUDA, OpenMP and SYCL translations with reference outputs; weak evidence for multi-file applications or other models such as HIP and Kokkos.
- Architecture and integration
- Orchestrator with translation, compile-diagnosis and execution agents, a repair-skill router, a Slurm GPU partition for execution, and an offline verifier that scores artifacts with a confidence value.
- Prerequisites
- Reference outputs and tests per kernel
- Slurm GPU partition and compilers
- Open-source model endpoint
- Repair cap and token budget
- Constraints
- Kernel-level benchmark only
- SYCL targets markedly weaker
- No performance evaluation
- Verifier is an LLM and can be wrong
- Security
- Unpublished or export-controlled codes need controlled model hosting and licensing review; the study used public kernels.
- Proof of value
- Port 5 to 10 local kernels and compare verified correctness with and without the offline verifier
- Count false passes found by reference-output comparison
- Record tokens and repair rounds per accepted port
Delivery
Role takeaway
- Work
- Select pilot kernels, build reference tests, configure the verdict-driven loop and verifier, run ports on the GPU partition, and have code owners review results.
- Dependencies
- Code owner time for review
- GPU partition and compiler access
- Reference test data
- Ownership
- The code's principal investigator owns acceptance; research computing owns the pipeline and environment.
- Skills and adoption
- Research software engineers move from writing ports to specifying tests and reviewing agent output and verifier confidence.
- Governance checkpoints
- Reference tests approved before runs
- Human review of low-confidence verifier judgments
- Record of models and compiler versions per port
- Acceptance criteria
- Proposed: accepted ports match reference outputs within agreed tolerance
- Proposed: zero stub or skipped-verification artifacts after review
- Proposed: review hours per accepted port recorded
- Risks
- Silent false passes such as placeholder code
- Weak SYCL translations
- Overreliance on verifier confidence
- Kernel results not carrying over to applications
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Separate translation, compile diagnosis and execution validation, route failures by verdict to targeted repair skills with a repair cap (9 iterations here), and add an offline verifier that audits artifacts for stubs, skipped checks and numerical error. Plan for a Slurm GPU partition and the CUDA, OpenMP and SYCL toolchains used in the study.
Governance
Who approves, reviews and stays accountable for outcomes?
Treat agent-ported code as unverified until domain owners check outputs; use the verifier's confidence scores only to prioritize human inspection, not to approve. Record which models and compilers produced each port and keep the original code as reference.
Security and privacy
What data, permissions and controls need testing?
The study uses public benchmark kernels and OpenAI-compatible model endpoints. Campus codes may be unpublished or export-controlled, so model hosting, data handling and licensing of sources would need review that the preprint does not address.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Shifts effort from manual porting toward writing tests and reviewing outputs; HPC specialists remain needed to judge numerical and performance behavior.
Procurement
What should contracts, pricing and exit terms secure?
When buying or building agentic porting tools, ask for independent verification of outputs, per-direction results rather than one average, and model-agnostic operation, since open-source models are a documented option.
Operating model
Which teams own the service once it runs?
Supports a research-computing service that hosts the pipeline and verifier while code owners remain accountable for acceptance of each port.
Publication history
- 2026-11-09Research Universities · Issue 623 resources
Stable resource ID: agent-skill-hpc-code-translation-cuda-openmp-sycl-llm-verifier-117-false-positives-20261106