Is 24GB of VRAM enough for running an LLM?
For inference, 24GB runs models up to about 8B parameters at 16-bit precision, a 13B model at 8-bit, and up to around 30B at 4-bit. A 13B model at 16-bit does not fit: its weights alone take about 26GB. It is not enough to fully fine-tune large models, and it will not hold a 70B model at usable precision without heavy quantization or splitting across cards. For most single-user inference and light fine-tuning, a 24GB card is a practical, cheap starting point.
In our latest check: At 4-bit with 4,096 tokens of context, 36 of the 46 models we estimate fit in 24 GB; the largest is Qwen2.5 32B at 20.9 GB. The cheapest 24 GB card to rent on demand as a single GPU is the A30 at $0.35 an hour (Massed Compute). A lower rate, $0.20 per GPU-hour for the RTX 3090 (TensorDock), is listed without a GPU count, so it is not counted here as a single-card rental.
The cheapest offer for each GPU with at least 24 GB on the card itself: on-demand rates first, then GPUs listed only on spot, community or marketplace terms.
| GPU | VRAM | $/GPU-hr | Where | |
|---|---|---|---|---|
| RTX 3090 | 24 GB | $0.20 | TensorDock on-demand | Rent → |
| A30 | 24 GB | $0.35 | Massed Compute on-demand | Rent → |
| RTX A6000 | 48 GB | $0.35 | Thunder Compute on-demand | Rent → |
| RTX 5090 | 32 GB | $0.35 | HyperAI on-demand | Rent → |
| RTX 4090 | 24 GB | $0.35 | TensorDock on-demand | Rent → |
| L4 | 24 GB | $0.44 | Jarvislabs on-demand | Rent → |
| RTX A5000 | 24 GB | $0.44 | Massed Compute on-demand | Rent → |
| RTX PRO 4000 Blackwell | 24 GB | $0.57 | RunPod secure cloud | Rent → |
| A40 | 48 GB | $0.59 | RunPod secure cloud | Rent → |
| Quadro RTX 6000 | 24 GB | $0.69 | Lambda on-demand | Rent → |
Cite this data
GPU Rental Prices, "Is 24GB of VRAM Enough for Running an LLM?", prices checked 21 November 2026, https://gpurentalprices.com/answers/is-24gb-vram-enough-for-llm
Quote it with a link to this page. Downloads of the whole dataset are on Data & API.
The rough rule is two bytes per parameter at 16-bit precision, one byte at 8-bit and half a byte at 4-bit, plus headroom for the key-value cache that grows with your context length. So a 7B or 8B model fits at 16-bit, a 13B model fits once quantized to 8-bit or 4-bit, and a 30B model can fit at 4-bit if you accept some quality loss. Long prompts and large batch sizes eat into that budget quickly; the model memory estimates add the cache for the context you choose.
Related questions
- What GPU do I need for a 70B model?
- What is the cheapest GPU for Stable Diffusion?
- What is the cheapest 24GB GPU to rent?
Numbers on this page come from our verified snapshot of 21 November 2026. Full table on the homepage; method in the methodology.