Glossary
The terms that decide what you actually pay, in plain English.
Effective cost per hour
Effective cost per hour is the real price you pay per GPU-hour once the extras are folded in, not just the advertised "from" rate. The headline number ignores things that hit your bill: minimum commitments, data egress, bundled storage, and how often a spot instance actually gets interrupted. Two providers can show the same sticker price and cost very different amounts in practice. We normalize for these so the numbers in our table are actually comparable across providers.
Related: Egress fees · Minimum commitment · Spot vs on-demand · GPU-hour
Egress fees
Egress fees are charges for moving data out of a provider's network, billed per gigabyte transferred. They matter for GPU work because model checkpoints, datasets, and outputs can run into terabytes, and on the big hyperscalers egress can cost more than the compute itself. Many GPU-focused clouds (RunPod and Lambda among them) charge nothing for egress, which is a real saving for data-heavy pipelines. Always check the egress rate before you commit, because it does not show up in the hourly GPU price.
Related: Effective cost per hour · Minimum commitment · GPU-hour
GPU-hour
A GPU-hour is one GPU running for one hour, and it is the unit almost all rental prices are quoted in. If you rent 8 GPUs for 3 hours, that is 24 GPU-hours. Prices are always per GPU, so an "8x H100" node listed at a given rate charges that rate for each of the eight cards. It is the base unit for comparing providers and for estimating what a training run or an inference month will cost.
Related: Per-second billing · Effective cost per hour · Multi-GPU node
Interruptible instance
An interruptible instance is a GPU rental the provider can reclaim at any time, usually with a short warning of seconds to a couple of minutes. It is the same idea as a spot instance: you trade guaranteed uptime for a lower price. To use one safely, your job needs to save progress to disk (checkpoint) so it can resume after being kicked off. Good for training runs, rendering, and batch inference; bad for anything a user is waiting on live.
Related: Spot vs on-demand · Secure cloud vs community cloud · Effective cost per hour
Minimum commitment
A minimum commitment (min-commit) is a floor the provider makes you agree to before you get a price or a machine, such as a minimum number of hours, weeks, or a reserved term. Reserved commitments lower the hourly rate in exchange for locking in spend, so a one-year reservation can be much cheaper than on-demand. The catch is you pay for the commitment whether you use it or not, which raises your effective cost if utilization is low. Cluster products in particular often carry weekly minimums that the advertised per-hour rate hides.
Related: Effective cost per hour · Multi-GPU node · GPU-hour
Multi-GPU node
A multi-GPU node is a single server holding several GPUs (commonly 8) wired together with a fast interconnect like NVLink or InfiniBand. The interconnect lets the GPUs share data quickly, which is what makes training a large model across all of them efficient. SXM cards and clusters are built for this; a bunch of separate single-GPU rentals are not, because they lack the fast link. If your model does not fit in one GPU's VRAM, a multi-GPU node is how you scale up.
Related: SXM vs PCIe · VRAM · Minimum commitment
Per-second billing
Per-second billing charges you only for the seconds a GPU is actually running, instead of rounding up to the nearest hour. It saves money on short or bursty workloads: a 90-second inference job costs 90 seconds, not a full hour. Providers vary in granularity, with per-second, per-minute, and per-hour all in use. For serverless and spiky traffic, finer granularity plus the ability to scale to zero is what keeps the bill low.
Related: GPU-hour · Effective cost per hour · Minimum commitment
Quantization
Quantization stores a model's numbers at lower precision (for example 8-bit or 4-bit instead of 16-bit), which shrinks how much VRAM the model needs. A smaller memory footprint means you can run the same model on a cheaper GPU, or fit a bigger model on the card you already have. In rental terms this is a direct cost lever: quantizing a model down can move it from an expensive 80GB card to a 24GB one at a fraction of the hourly price. The trade-off is a small, usually acceptable drop in output quality, which is why it is standard practice for inference.
Related: VRAM · Effective cost per hour · GPU-hour
Secure cloud vs community cloud
Secure cloud runs in vetted, datacenter-grade facilities with dedicated hardware and compliance controls, so it is more reliable and usually a bit more expensive. Community cloud pools GPUs from third-party and independent hosts, which lowers the price but makes availability and consistency more variable. The same GPU model is typically cheaper on a community tier, so the choice is a trade-off between cost and reliability. Use secure for production and sensitive data; use community to save money on experiments and interruptible work.
Related: Spot vs on-demand · Interruptible instance · Effective cost per hour
Spot vs on-demand
On-demand means you rent a GPU at a fixed hourly rate and keep it until you stop it. Spot (sometimes called interruptible or preemptible) means you rent spare capacity at a discount, but the provider can reclaim the GPU with little warning when someone pays more or demand rises. Spot is much cheaper and fine for work that can checkpoint and resume, like batch jobs or fault-tolerant training. On-demand is what you want for anything that must not be interrupted, such as a live inference endpoint or an interactive session.
Related: Interruptible instance · Effective cost per hour · Secure cloud vs community cloud
SXM vs PCIe
SXM and PCIe are two ways the same GPU chip gets packaged into a server. PCIe cards plug into a standard slot and talk to each other over the regular bus, which is fine for single-GPU work and cheaper to rent. SXM cards mount on a dedicated board with NVLink, giving much faster GPU-to-GPU bandwidth and higher power limits, which matters when you train across many GPUs in one node. For multi-GPU training the SXM version of a card (for example H100 SXM) is meaningfully faster than the PCIe version, and usually priced higher to match.
Related: Multi-GPU node · VRAM · GPU-hour
VRAM
VRAM is the memory built into a GPU, measured in gigabytes, and it sets the ceiling on how big a model you can run. Model weights, activations, and the key-value cache during generation all have to fit in VRAM, or the job fails to load or slows down badly. An H100 and an A100 both come in 80GB versions, while consumer cards like the RTX 4090 have 24GB. Picking a card is largely a question of finding the cheapest one whose VRAM is big enough for your model.
Related: Quantization · GPU-hour · SXM vs PCIe
Prices in action: the live table labels every row with its tier, and the methodology explains how we verify them.