When a team asks us about inference capacity, one of the first things our CEO, Jasper Zhang, asks back is simple: how many nodes are you running, and what’s your average utilization?
The answer usually follows a pattern. Teams find their busiest hour and reserve enough GPUs to serve it, which is the right call. For the rest of the day, some of those GPUs wait. Nobody planned it badly. Demand moves; reservations don’t. What that costs depends on how sharp your daily wave is and what you pay for the hours in the trough, and both have moved this year.
01 · The shape of demand
Inference traffic has a day shift. Reservations don’t.
In 2024, Microsoft published a week of production traffic from two Azure LLM services. We plotted it hour by hour against a reservation sized for the peak. [1]

The coding service is the extreme case. Its busiest hour carries 33 times the token volume of its quietest, and traffic all but disappears at night and over the weekend. If you use tokens as a stand-in for GPU load and size a flat reservation to the week’s peak, about two-thirds of that capacity would sit unused. The conversation service is steadier, with a 3x swing, but even there more than a third of peak capacity would go unused by the same estimate. These are estimates from token volume, not measured GPU utilization. [1, 2]
Other operators see it too. At Runway, traffic peaks around 9am ET and hits its low around 8pm, sometimes at less than half the morning load. Their engineering team summed up the bind in one line: “Provision for peak demand and most GPUs sit idle every night.” [3] Alibaba’s production models peak in the afternoon and fall off sharply in the early morning. [4] And a full year of traces from Chutes, an inference platform, shows the same daily and weekly cycles from start to finish. [5]
02 · Why you can’t just size down, and why idle hours show up anyway
Size for the trough and the morning breaks.
The obvious fix is to reserve less, but that just moves the pain to the morning. Too few GPUs at 9am means queues that back up and stay backed up. Shifting capacity around doesn’t save you either: moving nodes between clusters takes Runway 20 to 60 minutes, and their queueing model puts waits at roughly 10x service time once a pool runs at 90% utilization. Small pools need even more headroom. [3]
Training has its own version of the problem. In a 2026 Carnegie Mellon study of a GPU cluster, allocated GPUs with a program loaded sat at near-zero activity for 19.7% of execution time. [6] A training run finishes two days early. A team moves to B200s with months still left on its H100 reservation. Providers deal with the same thing on capacity they haven’t sold yet. It’s part of a bigger mismatch: GPUs are financed for years, while AI workloads run for weeks.
How big is the problem? Hard to say, because many teams don’t measure it. In an October 2026 VentureBeat survey of 128 organizations running GPUs, 57% reported production utilization at or below 50%, and a third didn’t track it at all. Among teams that reserve in advance, 31% said more than a quarter of their compute spend goes to capacity they reserved and didn’t use. [9]
03 · Why it costs more now
Idle hours have become more expensive.
For most of 2025, GPU prices fell. Then they turned. SemiAnalysis’s one-year H100 index climbed from $1.70 an hour in October 2025 to $2.35 by March 2026, and on-demand GPUs of every type were hard to find. [10]
SemiAnalysis’s public index stops in March. For a more recent read, Silicon Data’s H100 rental index, which tracks hourly rental prices across GPU clouds and marketplaces, was at $2.85 in October 2026. [15] Different indices measure different things, and we’ve written about the challenges of GPU rental indices before.
AWS raised Capacity Block prices in January and again in October. [11, 12] Nebius raised on-demand rates across H100, H200, B200 and B300 on October 1. [13]

Scarcity also explains why teams hang on to capacity they aren’t using. If you waited months for an allocation, giving it back means getting back in line.
04 · What GPU utilization really costs you
Every idle hour is billed to the hours you use.
The real price of a reserved GPU is the contract rate divided by the share of its reserved hours you actually use. The chart compares an H100 at the SemiAnalysis one-year index in October 2025 and March 2026, before and after the increase. [10]

At the March rate of $2.35, an H100 at 50% utilization costs $4.70 per useful hour, double the contract rate, and at 25%, $9.40. Back in October 2025, that same 50% cost $3.40. Same idle hours, 38% higher bill.
At October 2026’s market rate of $2.85, the math gives $5.70 per useful hour at 50% utilization, or $11.40 at 25%. [15]
Power barely registers. An idle H100 draws about 72 watts, roughly $0.14 of electricity a day. The rent on that same GPU at $2.85 an hour is $68.40. Leave an 8-GPU H100 node idle for a day, and you’ve paid $547.20 for capacity you didn’t use. [14, 15]
05 · What’s next
Reserve long. Give back what you don’t use.
Reserving long-term is still the right way to buy compute. It locks in supply and price. The catch is that capacity you don’t use stays stuck with you.
We’re starting with the idle capacity from section 2: the training run that wrapped early, or the H100s still on reservation after your team moved to B200s. With resell, you’ll be able to release whole nodes for 24 hours or more, with 48 hours’ notice. We resell those nodes on Hyperbolic, and you earn credit for the time they’re sold.
That doesn’t solve the daily wave yet. Hourly resale, the kind that could put tonight’s trough to work, is where we’re headed.
And if you locked in H100s near the $1.70 low in October 2025, the market now rents them for around $2.85 an hour. Your idle capacity may be worth more today than you paid for it.
There’s almost no public data on how reserved fleets are actually used hour by hour. We’d rather build this with the teams living it than guess, and we’ll publish what we learn in aggregate.
Design partner program: Help us build resell
We’re looking for a small group of teams with reserved GPU capacity on Hyperbolic, or bare-metal GPUs we can onboard onto our platform, to shape the program before it opens up.
You get: no resell fees during the pilot, early access, a say in how it works, and a shared Slack channel with the team building it
We ask: rough utilization data and an occasional feedback call
Apply to be a resell design partner →
Sources
Microsoft, Azure LLM Inference Trace 2024. Peak and idle figures are Hyperbolic’s analysis, using tokens as a proxy for GPU load and a flat reservation sized to the week’s peak.
Stojkovic et al., “DynamoLLM”, HPCA 2025 (Azure peak-to-valley ratios).
Runway, “Borrowing the Night: Reclaiming Idle Inference GPUs for Research”, July 2026.
Xiang et al., “ServeGen”, NSDI 2026 (Alibaba Bailian production traces).
Nixon et al., “A Year in LLM Serving”, 2026.
Lei et al., “The Energy Cost of Execution-Idle in GPU Clusters”, April 2026.
Meta, “The Llama 3 Herd of Models”, 2024.
Hu et al., “Characterization of Large Language Model Development in the Datacenter”, NSDI 2024.
VentureBeat Pulse Research, October 2026 (n=128; n=114 for reserving organizations).
SemiAnalysis, “The Great GPU Shortage: Rental Capacity”, April 2026. newsletter.semianalysis.com/p/the-great-gpu-shortage-rental-capacity
The Register, AWS EC2 Capacity Block price increase, January 2026.
VentureBeat, AWS Capacity Block price increases, November 2026.
Reuters, Nebius price increases effective October 2026.
NLR, idle H100 power measurement (72.5 W), April 2026. Electricity at $0.08/kWh, excluding cooling.
Silicon Data, H100 Rental Price Index (SDH100RT), October 2026. silicondata.com/products/silicon-index/h100. A standardized hourly rental price across neoclouds and GPU marketplaces, a different measure from SemiAnalysis’s one-year contract index. Used here as the most recent public reading.
:format(webp))
:format(webp))
:format(webp))
:format(webp))
:format(webp))
:format(webp))
:format(webp))
:format(webp))
:format(webp))
:format(webp))