OCP 2026: Run:ai Goes Fully Open Source
AI platform builders should focus on delivery and operations
Information current as of October 17, 2026
During OCP 2026, NVIDIA open-sourced the complete Run:ai platform and added network configuration, cluster acceptance, rack management, and bare-metal delivery components to the DSX OS infrastructure-operations layer. [1][2] Compared with the GPU scheduler already opened earlier, what expanded this year is the range of software enterprises can directly adopt, modify, and ship: a team can evaluate an entire platform, or plug capabilities such as organizational quotas and workload descriptions into an existing system.
These releases change the starting point for platform building in AI factories. For companies operating GPU clouds, model services, or enterprise AI platforms, the key questions become which ready-made capabilities to adopt, which controls to keep in-house, and who stays responsible for continuous delivery. The more complete the open foundation, the more carefully component capabilities must be separated from customer commitments.
A full platform release widens what can be adopted directly
On April 1, 2025, NVIDIA had already released the Apache 2.0-licensed KAI Scheduler for GPUs. Fair share, hierarchical queues, and gang scheduling — allocating resources for tasks that must start together as a group — were already open capabilities at that time. [13] The discussion should therefore start from what was already available: teams could adopt the scheduling engine, but still had to wire up organizational permissions, workload entry points, lifecycle management, and platform operations themselves.
The complete Run:ai repository puts the control plane, cluster services, installer, and clients together as a platform implementation that can be inspected and built. [4] This widens the scope of what is open and spares vendors and cloud providers some of the repeated work of assembling a product from components. For enterprises that already run a Kubernetes platform, the more useful part is that they can pull in only the missing pieces; for new operators without a platform yet, it adds a starting point that can be evaluated as a whole. The two adoption paths carry different migration risks, and "fully open source" alone does not mean an existing system must be replaced wholesale.
The independent components released this time cover four responsibilities: the KAI Scheduler, which decides where tasks are placed; KAI Resource Management, which expresses departments, projects, resource quotas, and node pools; the Workload Map, which describes how workloads are structured across frameworks; and the GPU Fractioning Operator, which connects CUDA-native GPU slicing to Kubernetes. [1] These are open-source software and interface contracts. Cross-vendor governance, compatibility testing, and long-term version commitments each still need to be verified separately; a release timed for OCP should not be read as a unified industry standard.
A more concrete change sits on the GPU-sharing path. Run:ai already had a proprietary slicing implementation; this announcement says it moves to CUDA 13.4 native capability, with the driver enforcing memory limits and the Operator providing Kubernetes integration and slicing metrics. [1][10] For platform owners this means the interfaces between driver, runtime, scheduler, and metering must be revalidated. An existing product name for "slicing" is no substitute for compatibility, interference, and recovery testing of the new implementation.
Two releases cover provisioning and operations
The complete Run:ai manages workloads on shared resources; this DSX OS addition covers the earlier stage of bringing a cluster into service. The Config Manager automates configuration of the switching network; the Cluster Readiness Engine runs stress workloads and checks cluster health; the Rack Management Service manages rack hardware and firmware; and the Cluster API Provider for NVIDIA Infrastructure Controller delivers bare-metal environments through a standard cluster lifecycle. [2] These were announced on October 12; the Run:ai full-platform open-source announcement came on October 13.
The value of these components is in removing manual steps that are hard to reproduce during delivery. A cluster that boots is not a cluster that has passed high-load acceptance; switch configurations that look identical do not guarantee that all nodes behave the same under real traffic. Turning configuration, stress testing, and hardware state into repeatable procedures is what makes it possible to distinguish equipment failure, software drift, and workload-specific problems. The efficiency gains still need to be measured in your own environment; a vendor's deployment-time claims cannot be turned directly into customer service commitments.
Connecting the two stages is what lets platform teams keep asking: what acceptance has this equipment passed, what resources can be committed right now, and who manages which workloads. The earlier stage reduces manual variance across repeated deployments; the later stage hands shared resources to specific teams and services. The components still need integration, but this year's releases give more of the pipeline a reference implementation that can be inspected and adopted.
Organizational quotas and workload description are the two key integration points
Resource allocation is first a question of rights, and only then a question of placement. How much guaranteed capacity a department gets, whether idle quota can be lent to another department, and how quickly borrowed quota must be returned all shape what the platform can promise to whom. KAI Resource Management expresses organizational hierarchy, quotas, and node pools separately, so platform teams can wire this logic into existing portals and approval flows. [8] That is not automatically contract billing, and it does not mean every quota percentage converts into the same share of online inference throughput.
On the workload side, a platform faces more than Deployments and Jobs. A single AI service may consist of a frontend, a prefill component that processes input, a decode component that generates tokens one by one, and auxiliary components, with different frameworks using different custom resources. The Workload Map provides a shared structure that helps tools understand components, roles, and state; its boundary is describing workloads, not doing queuing, autoscaling, or cross-cluster propagation itself. [9] Mistaking "unified description" for "unified execution" underestimates the control logic a platform still has to supply.
Adoption boundaries should be set at design time. An existing platform can keep its customer entry points, product catalog, and cost rules, and adopt the missing organizational or scheduling capabilities; a new platform can start from the full stack, then validate identity, network, storage, and operations dependencies item by item. The first path mostly evaluates interfaces and migration; the second mostly evaluates whole-stack maintenance and service responsibility. Both paths must measure the full time from a scaling decision to actually accepting traffic — component health is not a delivery result.
Build choices come down to continuous delivery and support responsibility
The first impact of this release shows up in software procurement and platform construction paths. Enterprises with existing Kubernetes and inference platforms can integrate by gap rather than migrate wholesale; new providers can evaluate the full platform, but should first validate with limited tenants, limited models, and explicit service-level objectives (SLOs). Choosing an open foundation does not require rewriting every algorithm — what enterprises genuinely need to own are the product catalog, admission rules, cost accounting, isolation tiers, and service responsibility.
Maturity also has to be judged per object. KAI Scheduler has a longer history of open use; the complete Run:ai distribution has just been published; the new DSX components each have different dependencies and operational boundaries. Source availability, code provenance from an existing product, and a vendor offering production support are three separate things to verify. The official notes state that the NGC Community Product requires no paid Run:ai subscription but does not include NVIDIA enterprise support, and the download remains governed by its terms. [1]
The full source and the independent components should each be adopted under their applicable licenses. [1][4] An open-source license decides how code may be used; distribution terms decide how packages are obtained; a support contract decides who responds to production incidents. Comparing the three separately is the only way to see whether software budgets and operations budgets actually fall. For an enterprise, what may be worth buying is long-term maintenance, compatibility validation, and incident support; what is worth owning in-house is the customer commitment and the acceptance evidence behind it.
The direction of this year's software news is now clearer: AI factories are getting more open components spanning infrastructure delivery through workload management, and vendors and operators can compose products around a common foundation. For a CTO, the decision is which stretch of the delivery pipeline to shorten with these open capabilities, and who stays accountable for versions, security, compatibility, and incidents over time. Only when those questions are answered does a wider scope of open software convert into stable, sellable token capacity.
Technical appendix
The following keeps the scheduling, isolation, metering, and failure details that need validation when adopting these components. They define the platform's delivery boundary; they are not a list of new features announced this year.
1 Aligning three scheduling layers with service objectives
Resource scheduling decides which GPUs tenants and workloads get; request scheduling decides which replica serves an inference call; in-engine scheduling decides how a replica batches and computes. The three layers operate on different time scales and different state. KAI's fair share, gang scheduling, and topology awareness solve one set of problems at the resource layer; Dynamo's cache-aware request routing (KV-aware routing) considers both cache reuse and already-assigned load at the request layer. [5][6] A replica with more cache hits is not necessarily the better choice if it is already congested.
Connecting the three layers requires stating the online-service constraints. GPU utilization answers how busy a device is within a sampling window; what users care about is whether requests succeed, when the first token arrives, how fast output continues, and whether the job finishes before its deadline. Larger batches or longer queues can raise device busyness while pushing user experience past its limit. The AIPerf performance tool counts effective throughput — goodput — as completed requests that meet a stated service objective, and keeps the attainment ratio as a second lens on quality. [3] An operations dashboard should read goodput, attainment ratio, and failure rate together.
A hypothetical shows the difference: same model, same request-length distribution and statistics window; option A completes 100 requests per second with 95% meeting the objective; option B completes 120 with only 70% meeting it. Goodput is then 95 versus 84 requests per second. This is not a product measurement. If failed requests are excluded from the attainment denominator, the numbers must be recomputed. B's total throughput is higher, yet it may be able to honor fewer contracts of equal grade.
For multi-tenant services, ingress rate limiting, routing, resource reclaim, and scaling must share compatible objectives. If the resource layer just reclaimed a replica while the request layer still forwards traffic to it, delivery breaks; if the ingress keeps accepting orders while the substrate cannot bring capacity up in time, it breaks too. Open components lower the cost of implementation — what operators must still own are the priorities and degradation rules between these controls.
Grove serves as background for this division of labor. It expresses a multi-role inference workload as one interdependent whole, then lets a scheduler place it; the project and its 2025 documentation are not new this year. [7] Once prefill and decode are disaggregated, scaling must also judge whether roles stay coordinated. Adding only input-processing capacity may just build up backlog in the generation stage faster; packing GPUs across longer distances may spend the compute time saved on KV transfers. A shared workload description makes these questions easier for tools to reason about, but ratios and placement still have to come from the real request distribution.
Service delivery thus gains one more time worth measuring: from the platform deciding to scale to the new capacity steadily accepting traffic. A pod running, weights loaded, communication established, caches warm, and ingress switched over do not all complete at the same moment. Procurement tracks GPU delivery speed; product teams must keep tracking this software delivery time, or hardware scale-up will not convert proportionally into sellable capability.
2 What GPU slicing and tenant isolation must still prove
A memory boundary is an important foundation for shared resources, but by itself it does not answer for latency interference between tenants, network permissions, storage access, or secrets management. GPU Fractioning's metrics help see low-level resource allocation; selling that share as a guaranteed service still requires testing neighbor workloads, bursts, reclaim, and restarts. [10] Development environments, small models, and strict-latency services do not necessarily fit the same sharing policy.
Run:ai's existing documentation includes identity, role-based access control (RBAC), project isolation, and audit capability. [11] After this open-sourcing, enterprises can more easily check how these capabilities attach to their existing identity and container environments — but they still need to assess them against their own threat model. An internal research team and an external untrusted tenant can differ enormously in isolation boundary, blast radius, and forensic requirements.
Product contracts must also state what is guaranteed. Dedicated equipment, reserved replicas, and guaranteed request performance are three different promises. Capacity borrowable at low peak must not be sold as always-available reserved capacity; reclaim time, recovery time, and warm-up needs belong in the internal capacity model. When a resource-allocation percentage cannot be mapped to a verifiable service outcome, the quota shown in a scheduler cannot carry a price.
3 How metering events enter cost and billing
Monitoring records what happened; billing enforces the contract. How input and output tokens are counted, how cache hits are credited, how cancellations, retries, and partial responses are charged — all need traceable rules. When model or tokenizer versions change, the corresponding basis should be retained. Slicing metrics help attribute resource cost, but they do not replace billing rules contracted per token, reserved throughput, or task outcome.
A platform should connect request identifiers, tenants, model versions, execution attempts, return status, and billing events. Monitoring may sample; billing must deduplicate, replay, and reconcile. Cost is also not just the GPU-seconds of successful requests: idle capacity held for a promise, model warm-up, failed retries, storage, network, and operations all belong in their service tiers. Otherwise a falling surface cost per million tokens may only mean more aggressive queueing or more sharing — not a better business outcome.
4 From cluster acceptance to incident handling
The Cluster Readiness Engine repository states its own boundary: it can run distributed workloads, flag anomalous nodes, and report causes — but it does not itself mark a node to stop receiving new scheduling tasks or add scheduling taints (cordon or taint). [12] Between "the checking tool is installed" and "a failed node stops taking orders," the upper platform still has to integrate admission control, capacity ledgers, and repair workflows. The cleaner product division is: the lower layer provides observation and execution; the upper layer decides when to stop admitting traffic, what happens to in-flight tasks, and when capacity is released again.
Rack management likewise cannot stand apart from scheduling policy. Firmware upgrades, node repairs, and full-rack maintenance all change available capacity; a platform that sees only total fleet size may keep counting resources under maintenance as committed. For a service provider, open software supplies the connective material — whether delivery stays reliable still depends on state propagation, responsibility boundaries, and failure drills.
Closing the loop on failures needs full accountability too. From detecting an anomaly, to stopping new traffic, to handling in-flight requests, to isolation, repair verification, and resuming sales, each step should leave evidence. Retries need budgets and idempotency boundaries, so one business request is not billed twice and remaining capacity is not crushed during an incident. DSX health checks and Run:ai workload management provide connectable pieces; whether user service degrades and recovers as promised still has to be proven by end-to-end drills.
References
[1] NVIDIA announcement of the full Run:ai platform open-sourcing, 2026-10-13.
https://developer.nvidia.com/blog/nvidia-runai-is-now-open-source-as-part-of-nvidia-dsx-ai-factory-platform/
[2] NVIDIA OCP 2026 announcement and new DSX OS components, 2026-10-12.
https://blogs.nvidia.com/blog/ocp-global-summit-vera-rubin-scale-ai-factories/
[3] AIPerf goodput definition and attainment ratio, retrieved 2026-10-17.
https://docs.nvidia.com/aiperf/tutorials/metrics-analysis/benchmark-goodput-with-ai-perf
[4] Official Run:ai code repository, retrieved 2026-10-17.
https://github.com/dsx-ai-factory/run-ai
[5] Official KAI Scheduler repository, retrieved 2026-10-17.
https://github.com/kai-scheduler/KAI-Scheduler
[6] Official Dynamo KV-aware routing documentation, retrieved 2026-10-17.
https://docs.dynamo.nvidia.com/dynamo/knowledge-base/concepts/system-architecture/kv-aware-routing
[7] Grove multi-component inference orchestration, 2025-11-10; architectural background, not listed as a release of this year.
https://developer.nvidia.com/blog/streamline-complex-ai-inference-on-kubernetes-with-nvidia-grove/
[8] Official KAI Resource Management repository, retrieved 2026-10-17.
https://github.com/kai-scheduler/kai-resource-management
[9] Official Workload Map repository, retrieved 2026-10-17.
https://github.com/dsx-ai-factory/workload-map
[10] Official GPU Fractioning repository, retrieved 2026-10-17.
https://github.com/kai-scheduler/gpu-fractioning
[11] Official Run:ai architecture and component documentation, retrieved 2026-10-17.
https://docs.nvidia.com/run-ai/self-hosted/documentation/getting-started/architecture
[12] Official Cluster Readiness Engine repository, retrieved 2026-10-17.
https://github.com/NVIDIA/cluster-readiness-engine
[13] NVIDIA first open-sources KAI Scheduler, 2025-04-01; used here as the historical baseline for this release.
https://developer.nvidia.com/blog/?p=98094