Chapter 4.88 of 9 in this part

Multi-tenancy and co-location

A model that needs 20 GB on an 80 GB card wastes three quarters of what you rented. Hardware partitioning fixes that with a guarantee; time-slicing fixes it with a hope. NVIDIA describes the failure mode precisely, and it is the exact resource this book says you are short of.

6 min read·revised 2026-08-08

Accelerators are sold whole. Workloads are not shaped whole. A model needing 20 GB on an 80 GB card leaves three quarters of a very expensive asset doing nothing, and no amount of batching fixes it because the constraint is that you rented an indivisible unit.

Multi-tenancy is the answer, and the choice of how you share determines whether you get a cost saving with a guarantee or a cost saving with a caveat.

The failure mode, in the vendor's own words

NVIDIA's description of what happens without partitioning is unusually blunt, and it names exactly the resource this book has spent Part 4 establishing as scarce:

Without MIG, different jobs running on the same GPU, such as different AI inference requests, compete for the same resources. A job consuming larger memory bandwidth starves others, resulting in several jobs missing their latency targets.

Read that against memory bandwidth is the product and the decode-is-memory-bound finding running through disaggregation and parallelism. The noisy neighbour does not steal your FLOPs. It steals your memory bandwidth — the one thing decode is already short of.

This is why co-location incidents present as latency mysteries. Nothing in your service changed; someone else's long-context request arrived.

What hardware partitioning actually buys

Multi-Instance GPU is a hardware split, not a scheduler policy:

MIG can partition the GPU into as many as seven instances, each fully isolated with its own high-bandwidth memory, cache, and compute cores … with guaranteed quality of service (QoS).

And the contrast with the cheaper approach is drawn explicitly:

With MIG, jobs run simultaneously on different instances, each with dedicated resources for compute, memory, and memory bandwidth, resulting in predictable performance with QoS and maximum GPU utilization. Unlike time slicing, each workload runs in parallel.

That word guaranteed is the whole commercial difference. Time-slicing and MPS also raise utilisation, and they are simpler to operate — but they share resources by scheduling rather than by dedicating them, so a neighbour's behaviour can still reach you. Hardware partitioning gives you a cost model you can put in a contract. Time-slicing gives you a cost model that holds until it doesn't.

If you are serving anyone else's traffic, or promising a latency SLO to an internal team, that distinction is the product.

Right-sizing is the actual saving

The headline number is utilisation:

With MIG, you can achieve up to 7x more GPU resources on a single GPU.

But the mechanism worth internalising is the flexibility, not the seven. NVIDIA's own GB200 example:

an administrator could create two instances with 93GB of memory each, four instances with 46GB each, or seven instances with 23GB each.

Match the slice to the model. A 7B model at 16-bit weights needs roughly 14 GB before KV cache — it fits a 23 GB slice with headroom and would waste most of a whole card. The saving is not "seven times more GPU." It is "stop buying 80 GB for a 14 GB model."

There is also a reconfiguration lever that is genuinely unusual in infrastructure:

seven MIG instances can be used during the day for low-throughput inference and reconfigured to one large MIG instance at night

The same physical card serves many small tenants during business hours and one large job overnight. That is a shape almost no other rented resource offers.

The cost of slicing finely, which is not zero

Take NVIDIA's three published GB200 profiles at face value and add them up:

Profile Per instance Total addressable
2 instances 93 GB 186 GB
4 instances 46 GB 184 GB
7 instances 23 GB 161 GB

The totals are not equal. Going from the two-way split to the seven-way split gives up about 13% of addressable memory — partitioning has overhead, and finer partitioning has more of it.

That is not an argument against MIG. It is an argument against reflexively choosing the maximum split: the right number of instances is the smallest number that fits your models, because every extra boundary costs a little of the thing you are trying to conserve.

The diagnostic

  1. What fraction of your GPU memory does your model actually use? If it is under half, you are paying for co-location capacity you have not claimed.
  2. Are you sharing by partition or by schedule? MIG dedicates; time-slicing and MPS arbitrate. Only one of them survives a hostile neighbour.
  3. Do you serve anyone you cannot phone? External tenants, or internal teams with SLOs, need the guarantee rather than the average.
  4. Have you sized the slice to the model, or taken the default? The profiles are the lever; seven is a maximum, not a recommendation.
  5. Does your load have a day/night shape? Reconfiguration between many-small and one-large is a real option and almost nobody uses it.
  6. When latency spikes with no deploy, do you check neighbours? Memory-bandwidth starvation from a co-tenant looks exactly like an unexplained regression.

What this chapter is not saying

It is not saying partition everything. A single large model that fills the card has nothing to gain and will pay the partitioning overhead for no benefit. And MIG is not universally available — it is a feature of specific data-centre GPU generations, so on other hardware the choice is between time-slicing and nothing.

It is saying that "one model per GPU" is a default, not a decision, and that on a fleet of 80 GB cards running 14 GB models it is one of the most expensive defaults available. The honest version of this chapter is short: measure your memory headroom, and if it is large, the question is only which sharing mechanism your promises can tolerate.

Sources & methodcaptured 2026-08-08

Sources, captured 2026-08-08: every quoted claim — the seven-instance maximum with each instance "fully isolated with its own high-bandwidth memory, cache, and compute cores," the "guaranteed quality of service (QoS)" framing, the without-MIG description of jobs competing for resources where "a job consuming larger memory bandwidth starves others, resulting in several jobs missing their latency targets," the "up to 7x more GPU resources on a single GPU" figure, the explicit "unlike time slicing, each workload runs in parallel" contrast, the GB200 profile example of 2×93 GB / 4×46 GB / 7×23 GB, and the day/night reconfiguration example — is quoted from NVIDIA's Multi-Instance GPU product page. The addressable-memory totals and the ~13% figure are computed by me from those three published per-instance capacities and verified in a separate pass; NVIDIA publishes the per-instance sizes but does not present the totals or comment on the difference, so the inference that finer partitioning costs addressable memory is mine, and it rests on treating the three profiles as alternatives on one physical GPU, which is how the source presents them. The ~14 GB figure for a 7B model at 16-bit weights uses the standard two-bytes-per-parameter approximation and excludes KV cache and activations, so it understates real requirements — the point it supports (that such a model fits a 23 GB slice and wastes most of an 80 GB card) survives that understatement comfortably. MPS is named but not sourced here: I have described time-slicing and MPS only through NVIDIA's own contrast with MIG rather than from their separate documentation, and no specific claim about MPS's isolation properties is made beyond that. No vendor pricing appears in this chapter — MIG is a partitioning capability rather than a priced SKU, and the hourly rates it lets you subdivide live in what an H100-hour costs. The diagnostic is my framing.

Want this done on your account rather than by you?

The handbook is the method, written out in full so you can run it yourself — that is the point of publishing it. If you would rather someone else did the first pass, the teardown is free and you keep the findings either way.