Best Cloud GPU for LLM Fine-Tuning
For LoRA and QLoRA fine-tunes of 7B to 13B models the value pick is an RTX 4090 at $0.35/hr: 24GB is enough once weights are quantized and only adapters are trained. Full fine-tunes and larger models need an 80GB A100 at $1.09/hr or an H100 at $1.99/hr for the extra bandwidth. VRAM here is set by the model size and the method, so decide LoRA vs full first, then pick the smallest card that fits.
In today's data: Lium, RunPod and Spheron rent all four picks, so one account covers the move from the RTX 4090 to the H200: on RunPod that is $0.89 to $3.79 an hour on-demand, or $0.34 to $3.59 on its community cloud, which can be interrupted.
RTX 4090 for LoRA on 7B to 13B
QLoRA on a 4-bit 7B to 13B model fits inside 24GB with room for a reasonable batch. Best dollar-per-token-trained for adapter tuning, as long as you do not need full-precision weights resident.
A100 for full fine-tunes
The 80GB A100 is the default for full fine-tunes of 7B to 13B and LoRA on larger models. Enough memory to hold weights, gradients, and optimizer state without aggressive offloading, at a rate well below H100.
H100 for the shortest runs
Higher memory bandwidth and FP8 support cut wall-clock time on full fine-tunes and shorten every epoch. Worth the premium when training time is the constraint or you are chaining many runs.
H200 for models over 80 GB
141GB of HBM3e lets you fine-tune 30B-class models on a single card and run long context without sharding. Pick it when a job simply will not fit in 80GB and you want to avoid multi-GPU complexity.
Worth knowing
- Full fine-tuning needs roughly weights plus gradients plus optimizer state, often 12 to 16 bytes per parameter with Adam, so a 7B full tune wants well over 80GB unless sharded.
- LoRA and QLoRA freeze the base weights and train small adapters, cutting memory dramatically: a 4-bit 13B QLoRA run fits in 24GB.
- Multi-GPU only helps if the interconnect is fast. On a single node prefer one large card over several small ones split by a slow bus.
- Gradient checkpointing and paged optimizers trade compute for memory and can drop a job onto a smaller, cheaper card.
FAQ
Yes, with QLoRA. Quantizing the base model to 4 bits and training only LoRA adapters fits a 7B, and often a 13B, inside 24GB. A full-precision full fine-tune of the same model needs an 80GB card or multiple GPUs.
An 80GB A100 handles most LoRA and mid-size full fine-tunes at a lower hourly rate. An H100 mainly buys speed through higher bandwidth and FP8, so choose it when shorter training time justifies the premium.
Providers counts every provider listing an offer that meets the pick's memory, in any billing class. 7 days is the average price move of the same on-demand configurations listed a week ago and today, not the change in the price shown. Prices render from today's verified snapshot, not from when this guide was written. Full table on the homepage; break-even math in the calculator.