Best Cloud GPU for LLM Training
Training or continued pretraining of an LLM is a multi-GPU job where interconnect and memory bandwidth decide cost, so the value pick is an 80GB A100 at $0.89/hr and the performance pick is an H100 at $1.99/hr. For the largest runs an H200 or B200 adds memory and bandwidth that shorten wall-clock time. Consumer cards are the wrong tool here because they lack the fast interconnect that keeps many GPUs fed.
The picks, with live prices
| Pick | GPU | VRAM | On-demand from | Where | |
|---|---|---|---|---|---|
| value pick | A100 | 80 GB | $0.89 | Jarvislabs on-demand | Rent → |
| performance pick | H100 | 94 GB | $1.99 | Voltage Park on-demand | Rent → |
| scale pick | H200 | 143 GB | $3.62 | Massed Compute on-demand | Rent → |
| frontier pick | B200 | 180 GB | $3.50 | Vultr on-demand | Rent → |
A100 value pick
The 80GB A100 remains the cost floor for serious training when you can get it on a node with fast interconnect. Lower hourly rate than H100 and plenty of proven throughput for models up to the tens of billions of parameters.
H100 performance pick
FP8, higher bandwidth, and fast NVLink or InfiniBand fabrics cut training time and often lower total cost despite the higher rate. The default for anything that runs for days across many GPUs.
H200 scale pick
141GB of HBM3e lets each GPU hold more of the model and a longer sequence, reducing sharding overhead and communication. Picks up where H100 gets memory-tight on large models or long context.
B200 frontier pick
Blackwell-class throughput and memory for the largest, most time-sensitive runs. Justified only when the job is big enough that faster silicon meaningfully shortens a multi-day training run.
Worth knowing
- Training from scratch is dominated by interconnect. A node with NVLink or InfiniBand keeps GPUs busy; a slow bus wastes the cards you are paying for.
- Rent by the cluster, not the card. Effective cost is total wall-clock time times node rate, so faster GPUs with better fabric often win even at a higher hourly price.
- Memory per GPU sets how much you must shard. More VRAM per card means less tensor and pipeline parallelism overhead.
- Confirm the offer is a real multi-GPU node with the stated interconnect, not several loose single cards billed together.
FAQ
Only small models or continued fine-tuning fit on one card. Pretraining a multi-billion-parameter model from scratch needs many GPUs with fast interconnect, so you rent a node or cluster rather than a single card.
Distributed training constantly exchanges gradients between GPUs. If the interconnect is slow the cards sit idle waiting on communication, so a node with NVLink or InfiniBand finishes faster and cheaper even at a higher hourly rate.
Prices render from today's verified snapshot, not from when this guide was written. Full table on the homepage; break-even math in the calculator.