GPU Rental Prices
SearchCtrl K
answered with live data · checked 21 November 2026

What GPU do I need to run Llama 405B?

Llama 3.1 405B has 405.9 billion parameters, so its 16-bit weights alone take about 812GB, more than an eight-card 80GB node holds (640GB). At full precision you need more memory than that, for example an eight-card node of 141GB H200 or 192GB MI300X cards, or two 80GB nodes working together. Meta also publishes an FP8 version of about 487GB, which fits on one eight-card 80GB H100 node with room left for the key-value cache. The table below lists the highest-memory GPUs we track for building such a node.

In our latest check: Llama 3.1 405B needs about 203 GB at 4-bit and 812 GB at 16-bit (weights alone, before cache and headroom). At 4-bit the weights alone fit on one 256 GB card, with the cache and headroom still to add; the cheapest rented as a single GPU on demand is the MI325X at $3.80 an hour (DigitalOcean). At 16-bit the cheapest node with enough combined memory for the weights is 8 × MI300X (1,536 GB combined) at $20.72 an hour (DigitalOcean). Combined memory still needs software that splits the model across the cards.

The cheapest offer for each GPU with at least 141 GB on the card itself: on-demand rates first, then GPUs listed only on spot, community or marketplace terms.

GPUVRAM$/GPU-hrWhere
MI300X192 GB$2.59DigitalOcean on-demandRent →
H200 (SXM)141 GB$3.20GMI Cloud on-demandRent →
MI325X256 GB$3.80DigitalOcean on-demandRent →
B200180 GB$5.00GMI Cloud on-demandRent →
MI350X288 GB$5.49RunPod secure cloudRent →
MI355X288 GB$6.20Daytona on-demandRent →
B300288 GB$7.40Hyperstack on-demandRent →
GB200186 GB$8.00GMI Cloud on-demandRent →
GB300288 GB$15.01Verda (DataCrunch) on-demandRent →
Rubin288 GB$24.36OVHcloud on-demandRent →
Cite this data

GPU Rental Prices, "What GPU Do I Need to Run Llama 405B?", prices checked 21 November 2026, https://gpurentalprices.com/answers/what-gpu-to-run-llama-405b

Quote it with a link to this page. Downloads of the whole dataset are on Data & API.

The memory math is two bytes per parameter at 16-bit: 405.9 billion parameters make about 812GB of weights, before the key-value cache that grows with context length and concurrent requests, and before the runtime's own working memory. An eight-card 80GB node has 640GB in total, so it cannot hold the 16-bit weights. Full precision needs a node with more memory per card, such as eight H200 (1,128GB) or eight MI300X (1,536GB), or a setup spanning two nodes, connected with NVLink or InfiniBand so the GPUs can share the model efficiently.

Quantization brings the number down. At 8-bit the weights take about 406GB; Meta's own FP8 release is about 487GB because some layers stay at 16-bit, and one eight-card 80GB H100 node holds it with room for the cache. At 4-bit the weights are about 203GB, so a node of four to six 80GB cards, or two to three MI300X at 192GB, can serve it with some quality tradeoff. Fewer, larger cards mean less interconnect overhead, which can matter for latency.

Because 405B is bandwidth and interconnect sensitive, the quality of the multi-GPU fabric matters as much as raw VRAM. The Llama 3.1 405B memory estimate gives the weights at each precision. The table below lists the largest-memory cards in our index, sorted by price, as building blocks for a node. For a full multi-card instance, see how node pricing works.

Related questions

Numbers on this page come from our verified snapshot of 21 November 2026. Full table on the homepage; method in the methodology.

Compare your offers

Saved prices keep their original dates. Confirm the configuration, terms and availability at the source.

Select up to three offers to compare their rates and terms.

Type a GPU, a provider, a model or a question.