Serverless vs Dedicated GPU Inference: How to Choose and What Each Costs per Million Tokens
The honest answer to "serverless or dedicated?" is that it depends on how busy you can keep a GPU. Serverless charges for the tokens you use. A dedicated GPU charges for the hours you hold it. Which one is cheaper comes down to utilization, and you can work out the break-even point with a few lines of arithmetic.
This guide shows how each option is billed on Nebula Block, how to calculate the break-even for your own workload, and what the math leaves out. Figures are as of November 2026. The throughput and serverless prices in the examples are hypothetical, so use your own numbers.
If your workload is an offline job rather than live traffic, our guide to batch vs serverless inference covers that case. This post is about choosing between paying per token and renting a GPU.
How each option is billed on Nebula Block
Serverless inference is pay-as-you-go. You call a model through an OpenAI-compatible endpoint at https://inference.nebulablock.com/v1, with no servers to provision and no models to load. Per-token prices are published on the pricing page and in the console. A $5 deposit (Tier 2) opens up the catalog, since the free tier has a daily request cap of 0 on most models.
GPU instances are billed by the hour, per configuration, for as long as the instance exists. Stopping one doesn't stop the charge; deleting it does. Renting a GPU needs Tier 3, which is a $10 deposit.
Serverless also comes with rate limits that depend on your tier:
Each model also has its own daily cap per tier, and the console's Limits page shows the exact numbers for your account. Tier 4 is reached by spending $100.
The math: when does a dedicated GPU win?
Three steps give you the break-even.
Step 1: what the GPU costs per month. Take the hourly price times 730 hours. Nebula Block's API docs include a sample price of $2.632 per hour for a single H100 PCIe VM in Canada. That works out to $1,921.36 per month if you keep it for the whole month. (It's a sample from the docs, not a quote. Check the console for real prices.)
Step 2: how many tokens it can produce. Tokens per second × 3,600 × 730 × utilization. Suppose, hypothetically, your model produces 1,500 tokens per second across all requests. At 100% utilization that's 3,942 million tokens a month.
Step 3: compare with the serverless price. The break-even is the monthly GPU cost divided by the serverless price per million tokens. The table below uses three made-up serverless prices to show how it plays out:
Read the last column as "how busy would I have to keep this GPU for it to beat paying per token?" If the serverless price is low, a dedicated GPU almost never catches up. If the serverless price is higher, or your model is expensive to serve, the bar drops quickly. Plug in the real per-token price of the model you'd actually use from the pricing page, and your own measured throughput.
The same formula also gives you cost per million tokens on the GPU side:
hourly price ÷ (tokens per second × 3,600 × utilization) × 1,000,000
Our guide to cutting inference costs covers how to raise the throughput and utilization inputs.
What the math leaves out
Rate limits. Serverless scales with your tier. A steady, high-volume workload can run into tokens-per-minute or daily caps, which is a practical reason to look at dedicated capacity.
Operations. A GPU you rent is a server you run: serving software, monitoring, upgrades, and on-call. Serverless has none of that. If nobody on your team wants to own it, price that in.
Model choice. Serverless gives you the catalog, which includes dozens of text and multimodal models. If you need your own fine-tuned model or a model that isn't listed, you need a GPU. For sizing, see the VRAM guide.
Control and latency. A dedicated GPU lets you tune the serving stack, and for latency-sensitive workloads you can go further with bare metal, which Nebula Block offers on H100. The Bare-Metal vs Cloud GPU Instances guide explains when that premium pays off.
Where the data goes. Models marked with a Canadian flag in the model catalog are hosted in Canada, and GPU instances can be deployed in Canada, the United States, Finland, France or Norway. If location matters, check the region of whichever option you choose.
Idle time. This is the big one for dedicated GPUs. An instance that sits at 10% utilization costs the same as one running flat out.
A practical way to choose
- Prototyping or unpredictable traffic: start with serverless. You pay only for what you use and there's nothing to set up.
- Steady traffic where measured utilization would clear your break-even: a dedicated GPU is worth pricing out.
- Your own or fine-tuned model: dedicated, since serverless only serves the catalog.
- A mix: many teams run baseline traffic on a dedicated GPU and send bursts to serverless, so they never pay for peak capacity around the clock.
Whatever you pick, measure before you commit. Rent a GPU for a day, run your real traffic, and compute cost per million tokens. A day of testing costs far less than a month of guessing.
The bottom line
Serverless is cheaper when your GPU would sit idle, and dedicated is cheaper when you can keep it busy. The break-even is a utilization number you can calculate: monthly GPU cost, divided by the serverless price, compared with what the GPU can actually produce. Do that arithmetic with your own figures before you choose.
Learn more at
- Email: contact@nebulablock.com
- Website: nebulablock.com
- Docs: docs.nebulablock.com
- Book a call: nebulablock.com/contact