{"resourceId":"atomicagent-bolt-e4b-vs-gemma4-e4b-local-agent-test-20261023","versions":[{"version":"external-c0efd6570095b4391b43fb89a91cf5f30d0a5984489521fff0455b6f2cee1b30","resource":{"id":"atomicagent-bolt-e4b-vs-gemma4-e4b-local-agent-test-20261023","title":"Atomic Agent test: Nemotron 3.5 Bolt E4B nearly ties its Gemma 4 E4B base on agent tasks (12/18 vs 11/18) but finishes far faster; the '4GB' GGUF file is 6.18 GB","organization":"Atomic Agent (independent local-agent developer); NVIDIA; Google","sector":"Independent technical evaluation (open models)","geography":"Global","publishedAt":"2026-10-23","publicationDate":"2026-10-23","eventDate":"2026-10-23","sourceName":"Atomic Agent blog","sourceLabel":"Independent benchmark write-up","sourceUrl":"https://atomicagent.io/blog/nemotron-3-5-bolt-vs-gemma-4-e4b/","evidenceClass":"independent-research","outcomeClass":"mixed","topics":["developers-agents","infrastructure"],"finding":"Hours after release, Atomic Agent ran Bolt E4B and its base Gemma 4 E4B on six offline agent tasks, three runs each, on one RTX 5090. Bolt passed 12 of 18 runs and Gemma 11 of 18, which the authors call close to a tie; Bolt finished all runs in 113 seconds versus 662, mostly because it generated about a tenth of the tokens. VRAM at 64K context was about 4.9 GB for Bolt and 4.3 GB for Gemma; NVIDIA's Q4_K_M file was 6.18 GB. Bolt ships Gemma 4's chat template under a Nemotron name, which caused their agent to fall back to a plain mode. Licences differ: OpenMDW-1.1 for Bolt, Apache 2.0 for Gemma.","sledRelevance":"Useful corrective for public-sector teams evaluating small local models: the independent test shows accuracy roughly equal to the free base model, with speed gains, licence differences and integration quirks that vendor pages omit.","evidence":"Small independent test (n=3 per task, six tasks, one GPU) by a developer with its own agent product; authors note one-run differences may be noise and that they used one sampling setting for all models rather than NVIDIA's recommended temperature.","architectureImplications":"Real memory use and file size exceed the '4GB' label; agent frameworks may need template handling updates.","governanceImplications":"Licence differs from the Apache 2.0 base model, which matters for public-sector open-source policies.","securityPrivacyImplications":"Test shows the agent framework blocking an attempt to overwrite an input file, a reminder that guardrails sit in the agent layer.","caveats":"Very small sample; single hardware setup; tester's own product; reasoning cap of ~500 tokens per step may affect reasoning-mode results.","streamIds":["nvidia"],"roles":{"sales":"Sales framing: an independent test suggests NVIDIA's new small model is about as accurate as its free base model but much faster on agent tasks. Interpretation: in public-sector conversations, position speed and token efficiency, not accuracy gains, and flag the licence difference. Do not cite the 4 GB figure as file size; the tested file was 6.18 GB. A bounded offer is a side-by-side evaluation on the customer's own tasks.","engineering":"Engineering framing: the test found near-equal pass rates, a large wall-time advantage from fewer tokens, about 4.9 GB VRAM at 64K context and a template mismatch that broke automatic mode detection. Pre-sales engineers should run their own tasks with NVIDIA's recommended sampling, check framework template handling and confirm licence acceptability. Treat the small sample as directional and verify on target hardware before recommending either model.","delivery":"Delivery framing: the template issue shows that new model releases can need integration fixes before they work properly in existing agent frameworks. Interpretation: delivery plans should include compatibility testing, a licence check and a larger evaluation set than this test's 18 runs. Run tests with the vendor's recommended sampling settings as well as defaults. Acceptance criteria below are proposals, not findings of the test."},"roleDetails":{"sales":{"customerProblem":"Teams need independent evidence before choosing between similar small local models.","stakeholders":["IT architects","Developers","Open-source policy owners"],"discoveryQuestions":["Is speed or accuracy the priority?","Which licences are permitted?","What hardware will run the model?"],"valueHypothesis":"Interpretation: faster small models could cut latency for local assistants; accuracy gains not shown.","potentialEngagement":"Side-by-side local model evaluation on customer tasks.","unsupportedClaims":["Bolt is more accurate than Gemma 4 E4B","Model file is 4 GB"]},"engineering":{"fit":"Local agent workloads on single GPUs.","architectureIntegration":"llama.cpp or similar serving with agent frameworks that detect chat templates.","prerequisites":["Template-compatible framework","Licence clearance","Representative task set"],"constraints":["Small sample","Single GPU test","64K context limit"],"security":"Keep file-write and command permissions in the agent layer.","proofOfValue":["Repeat test with more runs and recommended sampling","Measure VRAM on target devices"]},"delivery":{"work":"Run compatibility and accuracy testing before adoption.","dependencies":["Framework updates","Licence review"],"ownership":"Development team lead.","skillsAdoption":"Developers need evaluation methods for agent tasks.","governanceCheckpoints":["Licence approval","Evaluation sign-off"],"acceptanceCriteria":["Proposed: evaluation of at least several dozen runs per model","Proposed: template handling verified in chosen framework"],"risks":["Noise mistaken for improvement","Licence conflicts","Integration bugs"]}},"retrievedAt":"2026-10-24T02:56:54Z","enrichedAt":"2026-10-24T03:06:36Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Not addressed.","procurementImplications":"Licence terms (OpenMDW-1.1 versus Apache 2.0) should be reviewed before choosing between the derivative and its base model.","operatingModelImplications":"Small independent tests can be repeated in-house cheaply; agencies can run their own comparison before adoption.","sourceVerification":{"openedUrl":"https://atomicagent.io/blog/nemotron-3-5-bolt-vs-gemma-4-e4b/","referenceExcerpt":"With 18 runs per model, one run of difference can be noise. The speed gap is not noise: 113 seconds against 662.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}