OCP 2026: Helion and the Vera CPU Rack
Plan agent execution and model inference capacity separately
Information current as of October 17, 2026
OCP 2026 brought two distinct rack-design advances: AMD announced the Helion blueprint for agents, supporting a standalone CPU execution rack and an integrated enterprise CPU/GPU form; NVIDIA opened the Vera CPU rack design specification, extending ecosystem collaboration around an existing product. [9][1] Compared with provisioning everything around model inference, these two moves make the ability to execute code, call tools, and run sandboxes an infrastructure object that can be planned on its own.
The core change is that the execution pool gains a clearer product boundary: CPU, memory, and runtime can be configured to task demand. Delivery status still needs separating: Helion's partner configurations are in development, planned to be available in November and planned for OCP contribution; the Vera CPU and rack were announced at GTC in March, and this year's increment is the opening of the design specification. [9][10] They provide a construction path; actual task throughput still has to be proven end to end.
Helion lets execution and inference be provisioned separately
Helion's CPU-only execution rack is based on sixth-generation EPYC, with an announced top configuration of 32,768 CPU cores, positioned to give agent execution dedicated capacity and connect to separate model-inference infrastructure. The enterprise integrated form combines EPYC CPUs, Instinct MI350P GPUs, and general compute nodes in one blueprint. [9] Core count is a configuration ceiling — not the number of active agents, and not tasks completed per second; real applications are further bounded by memory, tool latency, and isolation approach.
The two forms map to different resource organizations. For operators with large model services already running and heavy demand for execution environments, a standalone CPU pool can scale with tool-call and code-execution pressure without buying GPUs every time. For enterprises deploying agents inside existing IT, the integrated form offers a more concentrated path to configure execution, inference, and auxiliary compute. Scaling separately and deploying centrally address different constraints; the choice should follow task structure, networking, cost, and operations.
The flexibility of a standalone execution pool is conditional. Between execution environments and model services sit network and state handoffs; queued model requests can leave many sandboxes waiting, while slow tool execution can leave model capacity periodically idle. The integrated rack removes some organizational complexity yet still faces CPU-to-GPU demand ratios shifting with the business. The value of productization is letting procurement, configuration, and operations work around these differences — not presuming a fixed core-to-GPU ratio.
AMD also mentioned a routing component — the token router — for choosing where model calls go, in the Helion release. [9] For product evaluation, what deserves checking is how model selection, quotas, billing, and failure fallback stay consistent. Different models' output lengths, tool-call counts, and failure-recovery behaviors shift CPU load. Evaluating routing capability and execution hardware fully separately risks missing the capacity impact of model-policy changes. Feature descriptions in the announcement are not a substitute for concrete versions and end-to-end tests.
What Vera opened this year is an existing CPU rack design
At GTC in March, NVIDIA announced a liquid-cooled rack integrating 256 Vera CPUs. October's OCP progress is bringing the Vera CPU rack design specification into the MGX open architecture expansion, explicitly serving tool calls, orchestration, and sandbox execution. [10][1] Ecosystem partners can now manufacture and integrate around an open design, and the CPU execution side shares rack-design principles with accelerated computing.
Placed next to Helion, this supports a more concrete trend judgment: agent execution is becoming an infrastructure capability that can be planned independently. After a model produces a plan, script, or action intent, environments still have to run programs, handle files, access services, and return results. Without a capacity and responsibility boundary for this stretch, faster GPU inference still leaves tasks stuck in execution queues.
Opening the rack design addresses part of the infrastructure collaboration; it does not automatically unify sandbox APIs, tool permissions, or task-acceptance standards. Racks can adopt shared power, cooling, and integration principles, while the application layer still decides how runtime, network egress, image management, and task state connect. For operators, an open design lowers some integration friction — but between a rack going live and "agents reliably finishing work" there remains a software and operations chain.
Planning a standalone execution pool starts from the full task path
Picture a data-analysis agent: it first understands the question, then reads files, runs code, checks results, possibly revises the code and retries, and finally writes the explanation. Model inference is only a few of those steps. File parsing, compression, database access, browser operation, and code execution draw separately on CPU, memory, storage, and external-service quotas. Which of them is the bottleneck depends on the task path — not on whether the product is classified "generative AI" or "agent."
Task-level tracing must therefore string model requests and tool calls together. A task can wait seconds in a sandbox queue while the model API's first-token metrics still look excellent; a tool timeout that triggers multi-round replanning raises total tokens. Externally the completion time stretches; internally an "inference throughput is growing" illusion can appear. Linking these events to one task identifier is what makes real cost and failure location visible.
Task decomposition also changes tail latency. With parallel tool calls, the gather phase follows the slowest branch; with sequential tool calls, startup, authentication, network, and result verification repeatedly enter the critical path. More parallelism also amplifies simultaneously arriving sandbox-creation requests. Agent platforms should therefore control both per-task parallel budgets and per-tenant concurrency so one task's self-acceleration does not crowd out others' completion.
CPU and memory need separate capacity math
CPU demand can start from per-task consumption before picking a processor. Orchestration-thread compute time, tool-process runtime, sandbox management, and network handling are best metered separately. Tasks waiting on external APIs may use almost no cores yet still hold memory, connections, sandbox slots, and state storage. Converting "active agents" directly into "busy cores" tends to overestimate compute and underestimate everything else.
A capacity-thinking example, nothing more: a system steadily receives 100 tasks per second, each consuming on average 0.3 core-seconds of CPU — mean compute demand is 30 continuously busy core-equivalents. Assume each task holds one sandbox, sandboxes last 2 seconds on average with 1 GiB resident, and there is no extra queueing: about 200 sandboxes are in flight, or about 200 GiB of memory. This is not a purchasing spec — bursts, tails, system overhead, redundancy, and per-core performance differences are all excluded. The example only shows CPU and memory do not scale by one shared multiplier.
Likewise, more cores do not guarantee a shorter critical path. Single-threaded tool speed, memory bandwidth, storage I/O, NUMA placement, and network round trips all matter. Sizing should use your own task set, watching which stage adding cores, raising clocks, adding memory, or improving storage actually moves. For a token-factory operator this means the CPU execution pool becomes a product capability with its own capacity, cost, and service objectives — not an unaccounted appendage to GPU servers.
Once execution is a standalone pool, the capacity model must join two arrival streams: user task arrivals and the rate at which models generate tool calls. At the same 100 tasks per second, if average tool calls per task rise, the execution pool can saturate before the model service; if tools shift to long waits on external APIs, more cores stop helping while sandbox-resident memory and connections keep growing. Model upgrades, prompt changes, and retry policy therefore belong in execution-side capacity regression tests.
Execution permissions are part of the capacity promise
A sandbox bounds what an execution environment can touch, but model-generated code can still do wrong things inside its permissions. Even without escaping to the host, it can wrongly modify mounted files or send inappropriate requests to permitted services. gVisor's security notes state plainly that isolation does not substitute for the security of the application inside the sandbox and does not remove access the application already holds. [6] For an agent product, tool permissions and action approval therefore need independent design.
Trade-offs among execution technologies should also follow the threat model. MicroVMs, application kernels, and ordinary containers differ in isolation mechanism, compatibility, and operations paths — they cannot be ranked by startup speed alone. The platform should validate the boundary among untrusted code, tenant data, host resources, and external systems, then measure the performance achievable under that boundary. Turning off egress restrictions or loosening resource controls for a better benchmark yields numbers that describe a different product.
Choose racks by effective tasks, not single peaks
An agent factory needs a denominator closer to business outcomes than token volume. Use "tasks completed within the agreed time, passing acceptance, with no constraint violations," broken out by task category. Code tasks can be accepted via tests and result checks; data-processing tasks via output completeness; tasks with external actions by verifying actual state. A model's own claim of completion cannot serve as acceptance for every task.
Cost per effective task should include model inference, CPU execution, memory residency, storage, external tool fees, failed retries, and any needed human review. For comparability, fix task difficulty, success conditions, and allowed retries. A cheaper model that induces more tool calls and repair loops can cost more overall; a faster CPU whose stage occupies almost no time may not deserve the premium. All of this should be verified by end-to-end records.
For the operator, whether a standalone execution pool is worth building starts from observing whether CPU execution and model inference peak at different times, whether their scaling needs diverge, and how much integration and operations work centralization saves. If tasks spend long stretches waiting on tools, CPU cores may not be the binding constraint — sandbox-resident memory, connections, and recovery capacity matter more. Quantify those demands first, then compare concrete racks and hosting options; only then does the procurement discussion share a denominator.
The commercial position of the CPU becomes clear: it carries the execution capability that turns model decisions into actual work. Enterprises should own task definitions, acceptance, permissions, cost accounting, and failure responsibility themselves; runtimes, template management, and basic orchestration can be integrated or bought. When buying managed services, the focus should be availability, recovery, audit, and data-handling boundaries — not environments created per second.
This year's moves by Helion and the Vera CPU rack give execution-side capacity planning clearer product objects. Enterprises can start from their own task paths, evaluate standalone pool versus integrated deployment, then choose processors, memory, runtimes, and services below. The technical route should still follow task structure and service requirements; a rack's name does not by itself produce a purchasing answer.
The important OCP 2026 change on this line is that execution capability is entering ecosystem collaboration as open blueprints and rack design specifications. What needs validating next is how these forms convert into stable supply, upgradeable software stacks, and services that can carry incident responsibility. GPUs still govern many inference stages; the execution pool must let tools and code do their permitted work — the number of effective tasks both support together is what most closely approximates the capacity an agent product can ultimately sell.
Technical appendix
The following expands startup, recovery, permissions, and acceptance methods for execution pools, using public runtime documentation and sandbox-provisioning tests. These implementations and micro-benchmarks are supplementary evidence — not proof of delivered capability in Helion or Vera.
1 Startup, warm pools, and pause/resume capacity boundaries
Sandbox startup includes at least environment preparation, runtime restore or boot, network and policy configuration, and the application actually being ready. CubeSandbox's official architecture lists control plane, node lifecycle, MicroVM, storage, and egress control separately — "creating an environment" is itself a chain of system calls. [3] Whether a create-API success means tool dependencies are ready must be confirmed per platform and test definition.
Warm pools can pre-stage popular environments and cut post-arrival waits, but holding environments consumes resources. How much to warm, how long to hold, and which tenants can share base templates should follow arrival distribution and isolation requirements. For occasional long tasks, keeping many warm instances may not pay; for short peaks, the cold-start curve after warm exhaustion matters more than the everyday average. Operators need to know whether the service promise covers the warm experience — or includes the experience during scale-out and recovery.
Pause and resume are another option, also with capacity commitments. CubeSandbox's lifecycle documentation separates freeing a paused environment's physical resources from retaining its scheduling quota: releasing more quota raises density but can cause later resumes to be rejected for lack of resources. [4] The design exposes a general problem: parking an environment converts immediate occupancy into future recovery demand — the demand does not disappear. Promising customers resume-anytime means reserving headroom for mass wake-ups.
Templates, images, dependency packages, and snapshots also push load onto storage. When many sandboxes resume at once, control-plane CPU may stay modest while snapshot reads, filesystem metadata, or network-policy updates queue. An operations dashboard should split create, ready, first-tool-execution, and reclaim times so not every wait gets blamed on the processor. Long sessions should also verify external connections and credentials after pause — restoring process memory does not guarantee the outside world has not changed.
2 The boundary of egress proxies and tool credentials
CubeSandbox's egress proxy lists domain control, credential injection, and audit as separate capabilities. [7] The division is instructive for platform design: keep tool credentials out of model-readable environments, scope access to task identity, and make unauthorized attempts leave evidence. But the proxy itself needs high availability, policy consistency, and a performance budget. Once a security function sits on the real request path, it should be load-tested with business traffic — not just checked "enabled" on an acceptance sheet.
Egress control must stay consistent with task identity. A resumed task may face expired credentials, changed external objects, or new access policy; restoring process memory does not mean external permissions should be restored as they were. Acceptance should include credential expiry, policy change, proxy unavailability, and unauthorized attempts, confirming failures are logged, denied, or safely retried.
3 What Intel's sandbox-provisioning micro-benchmark measures
Intel's Xeon test, published October 8, is a useful supplementary view of execution-pool provisioning — but it predates the summit and is not a Helion or Vera product test. The September CubeSandbox create/delete runs configured each sandbox with 2000 milliCPU and 2000 MiB, stepping up creation concurrency; near the 200-ms mean-create-latency threshold, concurrency was about 313, 277, and 246 for the Xeon 6990E+, Xeon 6980P, and EPYC 9965 respectively. The platforms differ in core count, memory speed, and power. [2]
Creation concurrency measures one segment of the provisioning path. It is not the number of continuously running agents, and mean latency does not imply tail-latency compliance. CubeSandbox's cube-bench documentation further notes that the test calls create and delete directly; its network-policy mode waits for HTTP creation success but does not check data-plane allow/deny or in-sandbox connectivity. [5] Control-plane performance, runtime security, and application success rate therefore need separate measurement, joined by one task chain. This test helps design acceptance methods; it should not sign end-to-end performance conclusions for any rack product.
4 Runtime roadmaps and product acceptance
Product teams can first build a small representative task set, then observe behavior under normal traffic, bursts, warm-pool exhaustion, and partial node loss. Errors and rejections must enter the statistics, and safe recovery after failure belongs in acceptance. CubeSandbox's public roadmap still lists failure recovery, scheduling-operations enhancements, and API compatibility as future work. [8] Available code, runnable examples, and required production commitments still have to be aligned item by item.
Procurement evaluation should keep three kinds of evidence apart: an open blueprint says how a system can be organized; a concrete configuration and support matrix says what can be purchased; an end-to-end task test says what results these resources deliver. Helion's planned availability cannot be written as in-stock; Vera's design opening this year does not by itself prove every partner configuration ships. Track concrete configurations, software versions, recovery capability, and service support — not peak core counts.
References
[1] Vera CPU rack role in NVIDIA's OCP 2026 announcement, 2026-10-12.
https://blogs.nvidia.com/blog/ocp-global-summit-vera-rubin-scale-ai-factories/
[2] Intel, Scaling Sandboxed Agentic AI on Xeon CPUs, 2026-10-08; date per Intel newsroom listing. Testing conducted September 2026.
https://www.intel.com/content/www/us/en/newsroom/news/data-center/scaling-sandboxed-agentic-ai-xeon-6.html
https://www.intel.com/content/www/us/en/newsroom/home.html?filters=[-7300596454%2C2661798488]
[3] CubeSandbox architecture documentation, retrieved 2026-10-17.
https://github.com/TencentCloud/CubeSandbox/blob/master/docs/architecture/overview.md
[4] CubeSandbox sandbox lifecycle and pause quotas, retrieved 2026-10-17.
https://github.com/TencentCloud/CubeSandbox/blob/master/docs/guide/lifecycle.md
[5] cube-bench official test notes and network-validation boundary, retrieved 2026-10-17.
https://github.com/TencentCloud/CubeSandbox/blob/master/examples/cube-bench/README.md
[6] gVisor security architecture and protection boundary, retrieved 2026-10-17.
https://gvisor.dev/docs/architecture_guide/intro/
[7] CubeSandbox security proxy documentation, retrieved 2026-10-17.
https://github.com/TencentCloud/CubeSandbox/blob/master/docs/guide/security-proxy.md
[8] CubeSandbox official public roadmap, retrieved 2026-10-17; roadmap items are not delivered capability.
https://github.com/TencentCloud/CubeSandbox/blob/master/docs/guide/roadmap.md
[9] AMD, Helion Agentic AI Compute at Rack Scale, 2026-10-12. The open blueprint is announced, with OCP contribution planned; partner configurations are in development and planned to be available November 2026 — not equal to OCP approval or broad off-the-shelf delivery.
https://ir.amd.com/news-events/press-releases/detail/1301/introducing-amd-helion-agentic-ai-compute-at-rack-scale
[10] NVIDIA announces Vera CPU and CPU rack at GTC, 2026-03-16. Historical baseline; the new OCP progress this year is the design-specification opening — the March product launch must not be re-dated as an October first.
https://nvidianews.nvidia.com/news/nvidia-launches-vera-cpu-purpose-built-for-agentic-ai
[11] Official AMD Helion product page and rack images, retrieved 2026-10-17. Images are official renders; delivered configurations follow partner offerings.
https://www.amd.com/en/products/rackscale-solutions/helion.html
The comparison of the two rack forms, task-capacity math, and procurement advice are the author's analysis. The 100 tasks/second, 0.3 core-seconds, 2-second residency, and 1 GiB figures are explicitly assumed illustrative values. The Intel vendor test supports only its sandbox-provisioning comparison under the stated configuration.