Can you add VRAM across rented GPUs?
Adding the memory capacities gives the physical total, but it does not prove that your application can use the memory as one pool. Check the application’s supported model-parallel or sharding method before selecting a multi-GPU node.
In today's data: Two 80 GB cards rented one at a time cost from $2.70 an hour today (2 × A100, Massed Compute); one MI300X with 192 GB on a single card costs from $2.59 (DigitalOcean). The single card avoids splitting the model at all. A lower rate, $1.09 per GPU-hour for the A100 (Thunder Compute), is listed without a GPU count, so it is not counted here as a single-card rental.
The model pages shortlist single-GPU memory scenarios. They do not promise that a model exceeding one card’s VRAM will run merely because several cards have enough combined memory.
Related questions
- What GPU do I need for a 70B model?
- What GPU do I need to run Llama 405B?
- Can you rent two GPUs instead of an eight-GPU node?
Numbers on this page come from today's verified snapshot. Full table on the homepage; method in the methodology.