// Chapter 06 · Deployment
Who Runs Open Models?
Publishing the weights lets others serve a model, but it does not decide who runs the inference, where it is processed or what it costs.
Data: OpenRouter providers and endpoints (9 November 2026, as displayed); providers’ price pages; GPU list prices; SemiAnalysis InferenceX; Hugging Face hub-stats.
39%
of the tokens routed through OpenRouter in a month went to independent hosts such as Together, DeepInfra or Wafer, which serve mostly open-weight models; labs serving their own models took 52% and the big clouds 9%.
OpenRouter providers page, 9 November 2026, as displayed.

Downloading a model’s weights does not mean running it on one’s own servers. The same open model can be reached through its developer’s API, an independent host, a big cloud or GPUs a company rents or owns. Choosing among them is a separate decision, with its own economics: the price of a token, the price of a GPU hour, how busy the GPUs are kept, the precision of the weights and the place where requests are processed.
01 Who serves
Independent hosts served 39% of the tokens routed through OpenRouter in a month
The lab that trains a model, the provider that serves it through an API, the operator that supplies the GPUs and the customer that pays need not be the same company. On OpenRouter, a router that sends requests to many hosts, labs serving their own models took 52% of the month’s tokens, independent hosts 39% and the big clouds 9%. Publishing the weights lets others serve a model, but they only do so where there is demand: the ten most used open models had a median of 17.5 providers each, and the median open model two.
Who served the tokens routed through OpenRouter in the month to 9 November 2026
Each square is one per cent of the month’s tokens
- 52% Labs serving their own models
- 39% Independent hosts
- 9% Big clouds
The five largest independent hosts, tokens in the month
Together
39.2 T tokens
United States
DeepInfra
33.7 T tokens
United States
Wafer
32.2 T tokens
United States
inference.net
31.7 T tokens
United States
DigitalOcean
20.4 T tokens
United States
Weights that anyone can download are not weights that most users run. In JetBrains’ 2025 survey, 70.5% of the developers who used DeepSeek reached it through DeepSeek’s own website or app, and 27.8% through its API.
The most used open models have a median of 17.5 providers; the median open model, two
99 verified open-weight models with an endpoint on OpenRouter, by how many providers offer them, 9 November 2026. In green, the ten with the most tokens from 1 to 14 October.
The ten most used: providers of each
- glm-5.3-flash36
- deepseek-v4.1-flash30
- deepseek-v4-flash-073126
- glm-5.224
- kimi-k321
- deepseek-v4-flash14
- mimo-v2.6-flash7
- hy4-preview7
- hy35
- nemotron-3-ultra-550b-a55b3
All open models
02 API or capacity
An API charges per token; dedicated GPUs charge per hour, busy or idle
That difference decides many cases. On their own price lists, many hosts of an open model charge what its lab does: 8 of the 9 that list Kimi K3 match Moonshot’s $3 and $15 per million tokens in and out. Renting GPUs gives control over the version, the precision and where requests are processed, but the hours are paid whether the GPUs work or not; owning them adds the hardware and the people who run it.
Three ways to run an open model
1
A shared API
a developer’s or a host’s endpoint
You pay
per token: Kimi K3 at $3.00 per million tokens in and $15.00 out, at Moonshot’s price
Someone else decides
the precision, the version, the rate limits and where requests are processed
The cost depends on
how many tokens you send
2
Dedicated capacity
GPUs a provider reserves for you
You pay
per GPU-hour, busy or idle: a B200 at a median $8.35 an hour on demand
Someone else decides
the hardware and the data centre; you choose the model, precision and serving setup
The cost depends on
how busy you keep the GPUs
3
Self-managed infrastructure
GPUs you own, or rent bare, and run yourself
You pay
for the GPUs (bought, or rented by the hour), power, space and the people who run them
Someone else decides
nothing: every choice and every failure is yours
The cost depends on
how busy you keep them, and for how long you keep them
How many hosts charge exactly what the developer charges, November 2026
At the developer’s own price At another price
Kimi K3 developer: $15.00 per M output tokens
GLM-5.3 developer: $4.40 per M output tokens
DeepSeek-V4-Pro developer: $3.96 per M output tokens
Qwen3.5-397B-A17B developer: $3.60 per M output tokens
Qwen3.8-27B developer: $3.00 per M output tokens
DeepSeek-V4.1-Flash developer: $1.20 per M output tokens
GLM-5.3-Flash developer: $0.50 per M output tokens
What tokens actually cost on one provider
Across the usage data Novita AI shared, the realized average price per million tokens fell 70.5% between June and October 2026, from $0.256 to $0.075. Tokens moved to cheaper models: the share on models with a realized price below $0.10 rose from 37.3% to 82.5%, while the average across models, each counted once, fell only 9.8%. Prices within that cheaper group fell as well, and the data do not separate how much of the drop each change explains. These are realized prices (revenue divided by tokens) on one provider, not list prices or a market-wide index.
Realized price per million tokens, per token and with each model counted once, and the share of tokens on models below $0.10, June–November 2026
Share of tokens on models below $0.10 per million
None of the three is cheaper in general; each fits a different situation. A shared API suits traffic that is still unknown or comes in bursts: nothing is paid when nothing is sent. Dedicated capacity suits a steady load that keeps the GPUs busy, or a team that needs a fixed version and precision. Running one’s own infrastructure makes sense when the data must stay in-house and there is a team to operate it. The routes can be combined: an API for the peaks, capacity of one’s own for the base load.
Comparing prices hides one trap. Most hosts charge what the lab charges, and some of the cheaper offers on OpenRouter are cheaper because they run the model at lower precision: the cheapest endpoints of Kimi K3 and GLM-5.3 declare 4 bits. Before the price, read the precision.
03 When it pays
When a rented node pays off against the API depends on the assumptions: from 26% of the month to never
An example. Renting eight B200s to run DeepSeek-V4-Pro costs about $48,700 a month, whether they work or not. Paying DeepSeek’s API instead costs nothing when idle and a fixed price per token. The node is cheaper only if it is kept busy: in this example, working flat out for at least 56% of the month. Below that, the API is cheaper. Change one assumption, such as the GPU price, the length of the prompts or the API price compared against, and that share moves a lot.
How much of the month eight rented B200s must work to cost the same as DeepSeek’s API for DeepSeek-V4-Pro, under each assumption
- The chapter’s scenario56%
- GPUs at the spot price33%
- GPUs at the highest list price96%
- 10 input tokens per output token26%
- 1 input token per output token84%
- Single-turn throughput77%
- API at the cheapest OpenRouter pricenever
Novita AI’s observed mix, about 21 input tokens per output token14%
The same calculation for other models
- Kimi K3 16 × B200not reached: needs 97 output tokens a second per GPU, measured 56
- GLM-5.3 4 × B200no independent measurement on this hardware
- Qwen3.5-397B-A17B 2 × B20068% of the month
- DeepSeek-V4.1-Flash 8 × B20039% of the month
- GLM-5.3-Flash 8 × B200not reached: needs 2,440 output tokens a second per GPU, measured 265
Per month, the rented node costs the same busy or idle; the API costs by the token
DeepSeek-V4-Pro on eight B200s at the median on-demand price, against what the API would charge for the tokens the node produces at its measured throughput, with three input tokens per output token.
The answer runs from 26% of the month to never. The share depends less on the model than on the assumptions: what the GPUs cost, how long the prompts are, how fast the model really runs and which API price it is compared with.
One observation moves the mix assumption. In the usage data Novita AI shared, input tokens were 95.4% of all tokens between June and November 2026, about 21 input tokens per output token, far from the three-to-one mix list prices are usually compared on. At that mix, the node in this example would cost the same as DeepSeek’s API at 14% of the month. It is one provider’s mix, and the measured throughput still comes from another workload.
04 The cost curve
The busier a rented node is kept, the cheaper each token; the API price does not move
A rented node costs the same every month. Spread over few tokens, each one is dear; over many, cheap. At full use, DeepSeek-V4-Pro on eight B200s produces a million output tokens for $4.43; DeepSeek’s own API charges $7.92 for the same mix, at any volume. The two lines cross at 56% of the month.
DeepSeek-V4-Pro on eight rented B200s: cost per million output tokens against the share of the month the node works
Assumptions
- The node is rented at the median public on-demand price, 730 hours a month.
- The API is the developer’s own list price; each output token comes with three input tokens.
- Throughput is an independent measurement (SemiAnalysis InferenceX) on agentic coding traces, with about 129 input tokens per output token, most of them cached. The two sides do not share a workload.
- Only the rented hardware is counted: no staff, storage, network, redundancy, setup or idle capacity.
Utilisation, the share of the month the GPUs are actually working, is the variable to watch: a node that sits idle at night or between bursts is paid for anyway. For two of the other models measured, the throughput is too low to reach the developer’s price at all. Owning capacity can pay for large and steady volumes, or when data must stay under the organisation’s control, but only a test on one’s own traffic can say where the line is.
05 Hardware and place
Memory, precision and place: what the price of a GPU hour does not say
Memory decides whether a model fits, bandwidth and compute how fast it runs, and the price what each hour costs. A B200 rents for nearly twice an H100, with more than twice its memory.
What a GPU rents for on demand: list prices of 26 October and 9 November 2026
Median price From the cheapest to the dearest list price
B200 180 GB
$8.35an hour
H200 141 GB
$6.15an hour
H100 80 GB
$4.46an hour
RTX PRO 6000 96 GB
$2.49an hour
For a European customer, place can weigh as much as price, and it is the hardest thing to check. Providers headquartered in the EU served 0.6% of the tokens on OpenRouter’s providers page, a share of one router’s traffic and not of inference in Europe. Alternatives exist: at least ten EU-based companies list this chapter’s Chinese open models in European data centres, mostly at or near the developer’s price, and their median on-demand H100 listed at $3.76 an hour. But a provider’s headquarters is not where it processes requests, and data residency is the provider’s own claim: ask where each request is processed.
06 Configurations
How many GPUs each open model needs, and what they cost a month
The largest open models need several GPUs, sometimes several servers. DeepSeek-V4-Pro is documented on eight H200s or eight B200s with its experts at 4 bits; Qwen3.5-397B-A17B on eight H200s at 8 bits or two B200s at 4 bits; only the dense Qwen3.8-27B fits on one card. Choose a model to see its servers and their monthly cost on demand.
Documented configurations: the GPUs that model makers and serving engines publish for each model, and their monthly cost on demand
Kimi K3 · 2.8 trillion parameters, 104 billion active
8 × B300
$53,900a month on demand
16 × B200
$97,500a month on demand
DeepSeek-V4-Pro · 1.6 trillion parameters, 49 billion active
8 × H200
$35,900a month on demand
8 × B200
$48,700a month on demand
GLM-5.3 · about 743 billion parameters, 39 billion active
4 × B200
$24,400a month on demand
8 × H200
$35,900a month on demand
DeepSeek-V4.1-Flash · 552 billion parameters plus a 196-billion memory, 8 to 16 billion active
4 × H200
$17,900a month on demand
8 × B200
$48,700a month on demand
GLM-5.3-Flash · 320 billion parameters, 18 billion active
4 × H200
$17,900a month on demand
8 × B200
$48,700a month on demand
Qwen3.8-27B · 27 billion parameters, dense
1 × RTX PRO 6000
$1,800a month on demand
1 × H100
$3,300a month on demand
07 Quantisation
The same model can be served at different precisions
Quantising stores a model’s weights in fewer bits: it needs less memory and runs faster, sometimes at a cost in quality. On OpenRouter, 43% of the endpoints of open models declare FP8 and 24% 4 bits; 25% declare nothing. Among the quantised copies people download from Hugging Face, GGUF leads, although the way the Hub counts its files inflates it.
The same model, three precisions: DeepSeek-V4-Pro’s 1.6 trillion weights in 16, 8 and 4 bits
Precision changes between hosts and between downloads: two populations, measured separately
Endpoints: the precision each host declares, 595 endpoints of open models on OpenRouter
Downloads: the format of quantised copies on Hugging Face, share of 30-day downloads
Two endpoints with the same model name can serve different precisions, and the price often follows the bits: the cheapest endpoints of Kimi K3 and GLM-5.3 on OpenRouter declare 4 bits. Before comparing two offers, check that they serve the same version at the precision the application needs.
Chapter 06 · Closing
How to decide where to run an open model
There is no single way to run a model whose weights are available. For a team that does not yet know its traffic, an API lets it start without committing to capacity. With heavy and predictable loads, a deployment of one’s own can pay, but the calculation depends on the throughput actually achieved, the price of the hardware and the work of running it. Before comparing two offers, check that they serve the same version at the precision the application needs, and ask where each request is processed.
- Traffic still unknown
- An API lets a team start without committing to capacity. Nothing is paid when nothing is sent.
- Heavy and predictable load
- A rented node run by the team itself can pay, but the calculation depends on the throughput actually achieved, the price of the hardware and the work of running it.
- Before comparing two offers
- Check that they serve the same version at the precision the application needs, and ask where each request is processed.