LLM VRAM requirements
Model weights are only part of GPU memory use. Compare weight sizes below, then open a model to account for context, runtime overhead and training memory.
Llama 3.1 405B
202.9GB
4-bit weights · 405.85B parameters
Check memory requirements →Qwen3 235B A22B
117.5GB
4-bit weights · 235.09B parameters
Check memory requirements →Mixtral 8x22B v0.1
70.3GB
4-bit weights · 140.63B parameters
Check memory requirements →Qwen2.5 72B
36.4GB
4-bit weights · 72.71B parameters
Check memory requirements →Llama 3.1 70B
35.3GB
4-bit weights · 70.55B parameters
Check memory requirements →Llama 3.3 70B
35.3GB
4-bit weights · 70.55B parameters
Check memory requirements →DeepSeek R1 Distill Llama 70B
35.3GB
4-bit weights · 70.55B parameters
Check memory requirements →Mixtral 8x7B v0.1
23.4GB
4-bit weights · 46.70B parameters
Check memory requirements →Qwen2.5 32B
16.4GB
4-bit weights · 32.76B parameters
Check memory requirements →Qwen2.5 Coder 32B
16.4GB
4-bit weights · 32.76B parameters
Check memory requirements →DeepSeek R1 Distill Qwen 32B
16.4GB
4-bit weights · 32.76B parameters
Check memory requirements →Qwen3 32B
16.4GB
4-bit weights · 32.76B parameters
Check memory requirements →Qwen3 30B A3B
15.3GB
4-bit weights · 30.53B parameters
Check memory requirements →gemma 3 27b
13.7GB
4-bit weights · 27.43B parameters
Check memory requirements →gemma 2 27b
13.6GB
4-bit weights · 27.23B parameters
Check memory requirements →Mistral Small 24B 2501
11.8GB
4-bit weights · 23.57B parameters
Check memory requirements →Qwen2.5 14B
7.4GB
4-bit weights · 14.77B parameters
Check memory requirements →Qwen2.5 Coder 14B
7.4GB
4-bit weights · 14.77B parameters
Check memory requirements →DeepSeek R1 Distill Qwen 14B
7.4GB
4-bit weights · 14.77B parameters
Check memory requirements →Qwen3 14B
7.4GB
4-bit weights · 14.77B parameters
Check memory requirements →phi 4
7.3GB
4-bit weights · 14.66B parameters
Check memory requirements →Phi 3 medium 4k
7.0GB
4-bit weights · 13.96B parameters
Check memory requirements →Mistral Nemo 2407
6.1GB
4-bit weights · 12.25B parameters
Check memory requirements →gemma 3 12b
6.1GB
4-bit weights · 12.19B parameters
Check memory requirements →gemma 2 9b
4.6GB
4-bit weights · 9.24B parameters
Check memory requirements →Qwen3 8B
4.1GB
4-bit weights · 8.19B parameters
Check memory requirements →Llama 3.1 8B
4.0GB
4-bit weights · 8.03B parameters
Check memory requirements →DeepSeek R1 Distill Llama 8B
4.0GB
4-bit weights · 8.03B parameters
Check memory requirements →Qwen2.5 7B
3.8GB
4-bit weights · 7.62B parameters
Check memory requirements →Qwen2.5 Coder 7B
3.8GB
4-bit weights · 7.62B parameters
Check memory requirements →DeepSeek R1 Distill Qwen 7B
3.8GB
4-bit weights · 7.62B parameters
Check memory requirements →Mistral 7B v0.3
3.6GB
4-bit weights · 7.25B parameters
Check memory requirements →gemma 3 4b
2.2GB
4-bit weights · 4.30B parameters
Check memory requirements →Qwen3 4B
2.0GB
4-bit weights · 4.02B parameters
Check memory requirements →Phi 4 mini
1.9GB
4-bit weights · 3.84B parameters
Check memory requirements →Phi 3 mini 4k
1.9GB
4-bit weights · 3.82B parameters
Check memory requirements →Llama 3.2 3B
1.6GB
4-bit weights · 3.21B parameters
Check memory requirements →Qwen2.5 3B
1.5GB
4-bit weights · 3.09B parameters
Check memory requirements →Qwen2.5 Coder 3B
1.5GB
4-bit weights · 3.09B parameters
Check memory requirements →gemma 2 2b
1.3GB
4-bit weights · 2.61B parameters
Check memory requirements →Qwen3 1.7B
1.0GB
4-bit weights · 2.03B parameters
Check memory requirements →DeepSeek R1 Distill Qwen 1.5B
0.9GB
4-bit weights · 1.78B parameters
Check memory requirements →Qwen2.5 1.5B
0.8GB
4-bit weights · 1.54B parameters
Check memory requirements →Qwen2.5 Coder 1.5B
0.8GB
4-bit weights · 1.54B parameters
Check memory requirements →Llama 3.2 1B
0.6GB
4-bit weights · 1.24B parameters
Check memory requirements →gemma 3 1b
0.5GB
4-bit weights · 1.00B parameters
Check memory requirements →Qwen3 0.6B
0.4GB
4-bit weights · 0.75B parameters
Check memory requirements →Qwen2.5 0.5B
0.2GB
4-bit weights · 0.49B parameters
Check memory requirements →Qwen2.5 Coder 0.5B
0.2GB
4-bit weights · 0.49B parameters
Check memory requirements →All model memory estimates
| Model | Parameters | 4-bit weights, GB | Cache estimate |
|---|---|---|---|
| Llama 3.1 405B | 405.85 billion | 202.9 | Not available |
| Qwen3 235B A22B | 235.09 billion | 117.5 | Config supported |
| Mixtral 8x22B v0.1 | 140.63 billion | 70.3 | Config supported |
| Qwen2.5 72B | 72.71 billion | 36.4 | Config supported |
| Llama 3.1 70B | 70.55 billion | 35.3 | Not available |
| Llama 3.3 70B | 70.55 billion | 35.3 | Not available |
| DeepSeek R1 Distill Llama 70B | 70.55 billion | 35.3 | Config supported |
| Mixtral 8x7B v0.1 | 46.70 billion | 23.4 | Config supported |
| Qwen2.5 32B | 32.76 billion | 16.4 | Config supported |
| Qwen2.5 Coder 32B | 32.76 billion | 16.4 | Config supported |
| DeepSeek R1 Distill Qwen 32B | 32.76 billion | 16.4 | Config supported |
| Qwen3 32B | 32.76 billion | 16.4 | Config supported |
| Qwen3 30B A3B | 30.53 billion | 15.3 | Config supported |
| gemma 3 27b | 27.43 billion | 13.7 | Not available |
| gemma 2 27b | 27.23 billion | 13.6 | Not available |
| Mistral Small 24B 2501 | 23.57 billion | 11.8 | Config supported |
| Qwen2.5 14B | 14.77 billion | 7.4 | Config supported |
| Qwen2.5 Coder 14B | 14.77 billion | 7.4 | Config supported |
| DeepSeek R1 Distill Qwen 14B | 14.77 billion | 7.4 | Config supported |
| Qwen3 14B | 14.77 billion | 7.4 | Config supported |
| phi 4 | 14.66 billion | 7.3 | Config supported |
| Phi 3 medium 4k | 13.96 billion | 7.0 | Not available |
| Mistral Nemo 2407 | 12.25 billion | 6.1 | Config supported |
| gemma 3 12b | 12.19 billion | 6.1 | Not available |
| gemma 2 9b | 9.24 billion | 4.6 | Not available |
| Qwen3 8B | 8.19 billion | 4.1 | Config supported |
| Llama 3.1 8B | 8.03 billion | 4.0 | Not available |
| DeepSeek R1 Distill Llama 8B | 8.03 billion | 4.0 | Config supported |
| Qwen2.5 7B | 7.62 billion | 3.8 | Config supported |
| Qwen2.5 Coder 7B | 7.62 billion | 3.8 | Config supported |
| DeepSeek R1 Distill Qwen 7B | 7.62 billion | 3.8 | Config supported |
| Mistral 7B v0.3 | 7.25 billion | 3.6 | Config supported |
| gemma 3 4b | 4.30 billion | 2.2 | Not available |
| Qwen3 4B | 4.02 billion | 2.0 | Config supported |
| Phi 4 mini | 3.84 billion | 1.9 | Not available |
| Phi 3 mini 4k | 3.82 billion | 1.9 | Not available |
| Llama 3.2 3B | 3.21 billion | 1.6 | Not available |
| Qwen2.5 3B | 3.09 billion | 1.5 | Config supported |
| Qwen2.5 Coder 3B | 3.09 billion | 1.5 | Config supported |
| gemma 2 2b | 2.61 billion | 1.3 | Not available |
| Qwen3 1.7B | 2.03 billion | 1.0 | Config supported |
| DeepSeek R1 Distill Qwen 1.5B | 1.78 billion | 0.9 | Config supported |
| Qwen2.5 1.5B | 1.54 billion | 0.8 | Config supported |
| Qwen2.5 Coder 1.5B | 1.54 billion | 0.8 | Config supported |
| Llama 3.2 1B | 1.24 billion | 0.6 | Not available |
| gemma 3 1b | 1.00 billion | 0.5 | Not available |
| Qwen3 0.6B | 0.75 billion | 0.4 | Config supported |
| Qwen2.5 0.5B | 0.49 billion | 0.2 | Config supported |
| Qwen2.5 Coder 0.5B | 0.49 billion | 0.2 | Config supported |
Estimate training cost · Compare LLM API pricing · GPU rental prices