Need to Talk to a Real Person? Call us at 877.378.3769.
Orchestration
Allocating a GPU and
placing a job are not
the same problem.
Kubernetes can now allocate accelerators properly. What it still can't do reliably is decide which accelerators — the ones with the right topology, the right thermal headroom, and no neighbour about to contend for the same NVLink domain. That gap between allocation and placement is where utilization quietly disappears, and it's the layer we build.
The Constraint
Three failure modes
that look like "we need more GPUs."
Each of these presents as capacity scarcity. None of them is. They're scheduling
problems wearing a procurement costume, and buying more hardware makes the bill
larger without making them go away.
Allocation ignores topology
Dynamic Resource Allocation solves how accelerators are requested and claimed. It does not, on its own, account for where they sit relative to each other. Bandwidth between GPUs inside one NVLink domain and across a slower boundary differs by more than an order of magnitude — so two valid allocations can perform nothing alike.
01
Multi-node NVLink jobs need coordination
Rack-scale systems let GPUs across chassis share a memory fabric, but that fabric has to be established for the set of nodes a job will use. Without that orchestration in the scheduling path, the jobs that most need rack-scale hardware are simply not schedulable on it.
02
Naive scoring picks GPUs that then fail
Rank candidates by adding up weighted signals and a GPU that scores well on seven dimensions outvotes one fatal deficiency on the eighth. It wins the placement, accepts the job, and then can't run it - the one outcome worse than declining to schedule at all.
03How we do it
Six layers,
assembled in order.
Deliberately built on upstream projects rather than around them. The proprietary work sits in exactly one place - the placement decision because that's the only layer where the open ecosystem still has a documented gap.
Profile the workload mix before choosing anything
Tightly coupled pre-training, elastic fine-tuning and latency-bound inference want different schedulers, different queueing semantics and different tenancy models. Committing to a control plane before this is characterised is how teams end up running two admission controllers over one capacity pool.
OUTPUT WORKLOAD PROFILE AND CONTROL-PLANE DECISION
Allocate through Dynamic Resource Allocation
DRA is now the supported path for accelerator allocation in Kubernetes, and the vendor driver work has moved into the upstream community. Building new clusters on the older device-plugin model means adopting a legacy interface on day one and migrating under load later.
OUTPUT DRA-NATIVE RESOURCE MODEL
Orchestrate the NVLink domain
For rack-scale systems, the memory-sharing domain spanning multiple nodes is established as part of scheduling rather than assumed to exist. This is what makes multi-node NVLink jobs actually placeable, and it's non-optional on current-generation rack hardware.
OUTPUT COMPUTE DOMAINS BOUND TO JOB TOPOLOGY
Place with gates, not a weighted sum
Health, topology fit and telemetry freshness act as multiplicative gates - fail one and the candidate is out, regardless of how it scores elsewhere. Headroom dimensions combine through a minimum so the scarcer resource dominates, and everything else is a tie-breaker.
OUTPUT GATED PLACEMENT ENGINE
Measure at the speed decisions are made
Standard exporter intervals leave telemetry tens of seconds stale - fine for dashboards, too slow to place against. We run a fast path for the signals that gate decisions and keep the slower aggregate path for reporting.
OUTPUT DUAL-PATH TELEMETRY WITH P95 HEADROOM
Reserve, then commit
A reservation ledger sits between decision and binding so two concurrent placements cannot select the same accelerator. Tenancy boundaries, quota and metering attach here - and we're explicit that partitioning for quality of service is not the same as isolating for security.
OUTPUT RESERVATION LEDGER AND TENANCY MODEL
Where The Value Sits
We Build on the Ecosystem
Everywhere Except One Layer.
Allocation, queueing, scheduling and domain orchestration are all solved upstream by teams larger than ours, and increasingly by a single converging roadmap.
Reimplementing any of it would be expensive and would age badly. The exception is the seam between them. Allocating an accelerator through one mechanism and placing it topologically through another leaves an accounting gap that the upstream projects themselves document as a known limitation. Closing that seam — correctly, under concurrency, with real telemetry — is the engineering that doesn't come out of a Helm chart.
What's included
What an AI Containers
Engagement Covers.
Cluster Build.
Kubernetes with DRA-native accelerator allocation from day one.
Domain Orchestration.
Multi-node NVLink domains established as part of scheduling.
Batch Scheduling.
A single admission path - gang scheduling, quota and preemption.
Slurm Interoperability.
HPC-style batch alongside Kubernetes where teams need both.
Placement Engine.
Gated, topology-aware scoring with fragmentation cost.
Reservation Ledger.
Concurrency-safe binding between decision and commit.
Telemetry.
Fast path for gating signals, aggregate path for reporting.
Tenancy and quota.
Namespace, partition and fair-share models per tenant.
Metering.
Attribution by accelerator or partition, with honest limits stated.
Health and Drain.
Automated draining and node replacement on degradation.
Control Planes
Pick one admission path. Running two is the common mistake.
Every option here is credible. What isn't credible is layering two of them over the same capacity pool and expecting quota to behave — a pattern we see often enough that it's the first thing we check on an existing cluster.
Kubernetes-native batch
Slurm on Kubernetes
Both, routed
Questions
Before You Commit
It's an architecture we deploy and operate, assembled from upstream components with our placement layer on top. You're not buying a black box, and you're not buying a pile of consulting hours either the cluster is yours, the components are open, and the parts we wrote are the parts nobody ships.
Yes, as long as the nodes meet standard modern accelerator and network fabric specifications.
Yes, through advanced GPU partitioning and time-slicing managed directly via our orchestration plane.
Metering tracks exact active utilization down to the resource slice assigned per container job.
Initial cluster assessment and core deployment typically range from 2 to 4 weeks depending on scale.
Bring us a cluster
Running at 40% and Let us Find out Why.
An assessment against your current scheduler, telemetry and job mix - before any discussion of more hardware.
Talk to an engineer