What is the difference between serverless GPU and instance pricing?
Instance pricing rents you a whole GPU by the hour for as long as you keep it running, idle time included. Serverless pricing bills only for the seconds your code is actually executing and scales to zero when idle. Serverless wins for spiky, low-duty workloads, while a dedicated instance wins for steady, high-utilization work.
With a rented instance you control the machine and pay a flat hourly rate whether it is busy or not. That is efficient when the GPU is loaded most of the time and predictable when you need a persistent environment, but you eat the cost of every idle minute.
Serverless platforms wrap the GPU behind a function or endpoint and charge per second of execution, often with a per-request component. You pay nothing when no requests arrive, which is ideal for bursty traffic, occasional batch jobs, or a demo that sits quiet most of the day. The tradeoffs are cold-start latency when a worker spins up and a higher effective per-second rate.
The rule of thumb: if your duty cycle is low, serverless usually costs less overall; if you would keep an instance busy for hours at a stretch, the dedicated instance is cheaper. Estimate your requests per day and their duration to see which side you fall on.
Related questions
- Should I look at GPU price per month or per hour?
- What is the difference between spot and on-demand GPU pricing?
- What is a GPU hour?
Numbers on this page come from today's verified snapshot. Full table on the homepage; method in the methodology.