H100 vs A100
The H100 is Nvidia's Hopper-generation flagship and the A100 is the previous Ampere generation it replaced. The H100 is faster at almost everything, especially large-model training and low-precision inference, thanks to higher memory bandwidth and an FP8 Transformer Engine the A100 lacks. The A100 is still a capable 80GB card and usually rents for less, which keeps it the value pick for many workloads. Compare the live rates above before deciding, since the gap between them moves daily.
H100
A100
Spec sheet
| H100 | A100 | |
|---|---|---|
| Architecture | Hopper | Ampere |
| VRAM (max variant) | 94 GB | 80 GB |
| FP16 TFLOPS (dense) | 835 | 312 |
| Memory bandwidth | 3900 GB/s | 1935 GB/s |
| TDP | 400 W | 300 W |
| Interconnect | NVLink 600GB/s | PCIe Gen4 |
| Launched | 2023 | 2020 |
Computed price-performance at today's cheapest on-demand rates: H100 $2.38 per PFLOP-hour vs A100 $2.85. Formula: cheapest $/hr ÷ dense FP16 PFLOPS (1 PFLOP = 1000 TFLOPS). Specs from vendor datasheets (source, source).
What they cost to rent (2026-07-08)
| H100 | A100 | |
|---|---|---|
| Cheapest on-demand $/hr | $1.99 | $0.89 |
| Cheapest spot $/hr | $1.40 | $0.68 |
| Cheapest source | Voltage Park | Jarvislabs |
| Providers offering it | 18 | 14 |
Cheapest verified on-demand rate per side from today's snapshot; highlighted = cheaper. Spot/marketplace tiers shown on the individual pages.
How they differ
| Dimension | H100 | A100 | Verdict |
|---|---|---|---|
| Architecture & generation | Hopper generation. Adds a fourth-generation Tensor Core and a dedicated Transformer Engine that accelerates the attention math behind large language models. | Ampere generation, one step older. Third-generation Tensor Cores, no Transformer Engine, and no native FP8 support. | The H100 is the newer architecture and wins on raw capability. The A100 is a proven older design that is still perfectly usable for most work. |
| VRAM & bandwidth | 80GB of HBM3 with roughly 3.35 TB/s of memory bandwidth, so it feeds the compute cores faster and keeps large models moving. | 80GB of HBM2e with roughly 2 TB/s of bandwidth. Same memory capacity, meaningfully less throughput. | Same 80GB ceiling, so both hold the same size model, but the H100's higher bandwidth is a large part of why it is faster in practice. If a model fits in 80GB on one it fits on the other. |
| Performance for training vs inference | Substantially faster for training large models and for inference at low precision, where the Transformer Engine and FP8 support cut memory use and boost throughput. | Strong at FP16 and BF16 training and inference, but with no FP8 path it cannot match the H100's low-precision speedups. | For frontier-scale training and high-throughput FP8 inference the H100 pulls clearly ahead. For mainstream fine-tuning and BF16 inference the A100 is still fast enough. |
| Price-performance for typical workloads | Rents around $1.99/hr at the low end. Costs more per hour but often finishes a given job faster, which can make it cheaper per unit of work on heavy training. | Rents around $0.89/hr at the low end, typically below the H100. For jobs that do not exploit FP8 or need the extra bandwidth, the lower rate wins on total cost. | Do the math on effective cost per hour times hours-to-finish, not the sticker rate. The H100 wins price-performance on large training; the A100 often wins on inference and smaller jobs where its lower rate dominates. |
| Availability breadth | Widely stocked across providers and tiers, but as the current flagship it is in higher demand and the cheapest listings sell out faster. | Very broadly available across nearly every provider, including cheap community and spot tiers, because it is a generation older and supply has caught up. | The A100 is easier to find cheaply and on interruptible tiers. The H100 is well stocked too, but the lowest prices come and go quickly. |
| When the older card wins | The H100 is overkill and overpriced for workloads that never touch FP8 or saturate its bandwidth, where you pay a premium for speed you cannot use. | The A100 wins for BF16 inference, small-to-mid fine-tunes, development and prototyping, and any budget-bound job that can trade some speed for a lower hourly rate. | If your workload does not specifically benefit from Hopper's FP8 and bandwidth, the A100 is the smarter rent. Reach for the H100 only when the job actually uses what it charges for. |
Pick H100 when
- You are training or fine-tuning large models and want the fastest wall-clock time
- Your inference stack uses FP8 and benefits from the Transformer Engine
- The job is bandwidth-bound and finishes fast enough to offset the higher hourly rate
Pick A100 when
- You are doing BF16 inference, prototyping, or mid-size fine-tunes that do not need FP8
- You want the lowest hourly rate and can find it on a community or spot tier
- Availability and budget matter more than squeezing out the last bit of speed
FAQ
Only when your workload uses what it charges for. On large-model training and FP8 inference the H100 can finish faster and cost less per unit of work despite the higher hourly rate. On BF16 inference and smaller jobs the A100's lower rate usually wins.
The common datacenter versions of both come with 80GB, so they hold the same size model. The difference is memory type and bandwidth: the H100 uses faster HBM3 at roughly 3.35 TB/s versus the A100's HBM2e at roughly 2 TB/s.
No. FP8 and the Transformer Engine are Hopper-generation features exclusive to the H100. The A100 runs FP16 and BF16 well but has no native FP8 path, which is a key reason the H100 is faster for low-precision inference.
Prices render from the live dataset each build. More: the screener · price movers · methodology.