Spot vs On-Demand vs Serverless GPU Pricing: When Each Wins

The short version: rent on-demand when the job cannot stop, rent spot or community capacity when it can checkpoint and resume, and use serverless when traffic is bursty enough that paying per second of actual use beats keeping a GPU warm. Most comparison sites blend these three into one price column. We keep them separate everywhere on this site, including serverless as its own kind rather than a cheaper-looking hourly rate, because to our knowledge no other GPU comparison site does, and mixing them produces numbers you cannot act on.

Here is each kind exactly as our tables define it.

On-demand and secure: the firm price

On-demand (and secure cloud, the datacenter tier of marketplace providers like RunPod) is a firm, fixed price for capacity that is not interruptible: you keep the GPU until you release it, at the rate you saw when you started. This is the honest baseline for comparing providers, which is why our price tokens, records, movers, and the GPU Price Index are all computed from firm prices only. Today's cheapest firm H100 rate, for example, is $1.99/hr, and that number is live, not from the day this guide was written.

Pay the firm price for production endpoints, deadline work, long uncheckpointed jobs, and anything where an interruption costs more than the discount saves.

Spot and community: the interruptible discount

Spot capacity is spare inventory rented at a discount in exchange for the provider's right to reclaim it, sometimes with minutes of notice. Community capacity is the marketplace variant: GPUs from independent hosts, including networks of consumer PCs like Vast.ai, RunPod Community Cloud, and SaladCloud. Our tables group both as interruptible, because that is what they have in common: the price is real, the availability is not guaranteed. See spot vs on-demand and secure vs community in the glossary.

How big is the discount? It moves daily and differs per card, so we do not quote a fixed rule here. Every GPU page on this site computes the live gap between the cheapest interruptible and cheapest on-demand rate for that card from today's snapshot; check the card you actually want, for example H100 or RTX 4090. It is also not guaranteed to be a discount at all: on thin inventory, a spot rate occasionally sits above the best on-demand rate for the same card, which is exactly the kind of thing a live index exists to catch.

Rent interruptible capacity for checkpointed training, batch inference, rendering queues, and experiments, any job where "resume from the last checkpoint" is a non-event.

Serverless: per-second, no VM

Serverless on this site means dedicated GPU hardware billed per second of execution, with no VM access: you deploy a container or a model, the platform runs it, and you never hold an instance. That is fal, Modal, Replicate, and Fireworks, plus the serverless products of instance providers like RunPod. It is not an hourly rental at all, which is why we refuse to blend it into hourly tables: in our data, serverless per-second rates for a given card typically work out above the same card's on-demand hourly rate. That premium is rational. You are paying for scale-to-zero, and if your GPU would sit idle most of the hour, the more expensive second is the cheaper month.

The full per-second table lives on the serverless page. Scope note: this covers renting serverless GPU hardware, not per-token LLM APIs, which are a different product we deliberately do not compare.

The decision in three questions

1. Can the job survive an interruption? If a kill mid-run means restarting from zero, buy firm capacity. If it means resuming from a checkpoint, take the interruptible discount.
2. What shape is the work? Continuous and predictable favors on-demand. Long but pausable favors spot. Spiky, idle-dominated inference favors serverless.
3. How much ops effort will you spend? Spot needs checkpointing, retry logic, and tolerance for host variance. Serverless needs almost nothing but accepts cold starts and the premium rate. On-demand is the zero-surprises middle.

Common traps

• Spot for long uncheckpointed training. The classic. One reclaim near the end of a multi-day run costs more than the discount saved. Checkpoint first, then go interruptible.
• Serverless bill surprises. Per-second pricing meters more than the GPU: cold starts, and on some platforms CPU, RAM, or per-request components, are billed too. A steady endpoint on serverless can quietly cost multiples of an on-demand instance doing the same work. Estimate at your actual utilization before committing.
• Treating community hosts as datacenter hosts. Community prices are per host, and hosts differ in bandwidth, uptime, and hardware condition. The headline "from" price is the best host, not the typical one.

How the kinds appear on this site

Every offer row carries its kind badge: on-demand, secure cloud, community cloud, spot / interruptible, or serverless. The all-GPUs table and the VRAM pages show two price columns, "On-demand" and "Spot/community from", and that second aggregate deliberately excludes serverless, so an interruptible "from" price is never secretly a per-second serverless rate. Serverless gets its own page and its own section on each GPU page. Firm-price surfaces, movers, records, and the GPU Price Index, are computed from on-demand and secure offers only.

FAQ

Is spot always cheaper than on-demand?

Usually, not always. Spot prices float with marketplace supply, and our snapshots occasionally record a card whose cheapest spot rate sits above its cheapest on-demand rate. Check the live gap on the GPU page for your card before assuming the discount.

What is the difference between spot and community?

Spot is a provider reclaiming its own spare datacenter capacity; community is capacity from independent hosts on a marketplace, consumer PCs included. Both are interruptible, which is why our tables group them, but community adds host-to-host variance on top of interruption risk.

Why is serverless more expensive per hour than on-demand?

Because you are not buying hours. Serverless bills only the seconds your code runs and scales to zero between requests, so the platform carries the idle risk and prices that into the rate. At low utilization you still come out ahead; at sustained high utilization an on-demand instance wins.

Can I mix kinds in one project?

That is usually the right answer: train on spot with checkpoints, serve the stable production load on-demand, and put the spiky overflow or the demo endpoint on serverless. The kinds are tools, not camps, and the live table prices all three side by side per GPU.