Best Cloud GPU for LLM Training
Training or continued pretraining of an LLM is a multi-GPU job where interconnect and memory bandwidth decide cost, so the value pick is an 80GB A100 at $1.09/hr and the performance pick is an H100 at $1.80/hr. For the largest runs an H200 or B200 adds memory and bandwidth that shorten wall-clock time. Consumer cards are the wrong tool here because they lack the fast interconnect that keeps many GPUs fed.
In our latest check: Cerebrium, CoreWeave, Google Cloud, Hyperstack, Koyeb, Lium, Modal, Oracle Cloud, RunPod, Spheron and Verda (DataCrunch) rent all four picks, so one account covers the move from the A100 to the B200: on Hyperstack that is $1.35 to $6.00 an hour on-demand.
A100 for the lowest hourly rate
The 80GB A100 remains the cost floor for serious training when you can get it on a node with fast interconnect. Lower hourly rate than H100 and plenty of proven throughput for models up to the tens of billions of parameters.
H100 for runs that last days
FP8, higher bandwidth, and fast NVLink or InfiniBand fabrics cut training time and often lower total cost despite the higher rate. The default for anything that runs for days across many GPUs.
H200 for long context and large models
141GB of HBM3e lets each GPU hold more of the model and a longer sequence, reducing sharding overhead and communication. Picks up where H100 gets memory-tight on large models or long context.
B200 for the largest runs
Blackwell-class throughput and memory for the largest, most time-sensitive runs. Justified only when the job is big enough that faster silicon meaningfully shortens a multi-day training run.
Worth knowing
- Training from scratch is dominated by interconnect. A node with NVLink or InfiniBand keeps GPUs busy; a slow bus wastes the cards you are paying for.
- Rent by the cluster, not the card. Effective cost is total wall-clock time times node rate, so faster GPUs with better fabric often win even at a higher hourly price.
- Memory per GPU sets how much you must shard. More VRAM per card means less tensor and pipeline parallelism overhead.
- Confirm the offer is a real multi-GPU node with the stated interconnect, not several loose single cards billed together.
FAQ
Only small models or continued fine-tuning fit on one card. Pretraining a multi-billion-parameter model from scratch needs many GPUs with fast interconnect, so you rent a node or cluster rather than a single card.
Distributed training constantly exchanges gradients between GPUs. If the interconnect is slow the cards sit idle waiting on communication, so a node with NVLink or InfiniBand finishes faster and cheaper even at a higher hourly rate.
Providers counts every provider listing an offer that meets the pick's memory, in any billing class. 7 days is the average price move of the same on-demand configurations listed a week ago and today, not the change in the price shown. Prices render from today's verified snapshot, not from when this guide was written. Full table on the homepage; break-even math in the calculator.