// Chapter 06 · Deployment

Who Runs Open Models?

Publishing the weights lets others serve a model, but it does not decide who runs the inference, where it is processed or what it costs.

Data: OpenRouter providers and endpoints (9 November 2026, as displayed); providers’ price pages; GPU list prices; SemiAnalysis InferenceX; Hugging Face hub-stats.

39%

of the tokens routed through OpenRouter in a month went to independent hosts such as Together, DeepInfra or Wafer, which serve mostly open-weight models; labs serving their own models took 52% and the big clouds 9%.

OpenRouter providers page, 9 November 2026, as displayed.

Illustration: layered landscape with a pale sun
Deployment

Downloading a model’s weights does not mean running it on one’s own servers. The same open model can be reached through its developer’s API, an independent host, a big cloud or GPUs a company rents or owns. Choosing among them is a separate decision, with its own economics: the price of a token, the price of a GPU hour, how busy the GPUs are kept, the precision of the weights and the place where requests are processed.

01 Who serves

Independent hosts served 39% of the tokens routed through OpenRouter in a month

The lab that trains a model, the provider that serves it through an API, the operator that supplies the GPUs and the customer that pays need not be the same company. On OpenRouter, a router that sends requests to many hosts, labs serving their own models took 52% of the month’s tokens, independent hosts 39% and the big clouds 9%. Publishing the weights lets others serve a model, but they only do so where there is demand: the ten most used open models had a median of 17.5 providers each, and the median open model two.

Who served the tokens routed through OpenRouter in the month to 9 November 2026

Each square is one per cent of the month’s tokens

The five largest independent hosts, tokens in the month

  1. Together

    39.2 T tokens

    United States

  2. DeepInfra

    33.7 T tokens

    United States

  3. Wafer

    32.2 T tokens

    United States

  4. inference.net

    31.7 T tokens

    United States

  5. DigitalOcean

    20.4 T tokens

    United States

Monthly tokens shown on OpenRouter’s providers page on 9 November 2026, open and closed models together, grouped by us: labs serving their own models (OpenAI, Tencent, DeepSeek…), independent hosts that serve other labs’ models, and the big clouds (Google Vertex, AWS, Azure). Tiles are sized by tokens. Source: OpenRouter.

Weights that anyone can download are not weights that most users run. In JetBrains’ 2025 survey, 70.5% of the developers who used DeepSeek reached it through DeepSeek’s own website or app, and 27.8% through its API.

The most used open models have a median of 17.5 providers; the median open model, two

99 verified open-weight models with an endpoint on OpenRouter, by how many providers offer them, 9 November 2026. In green, the ten with the most tokens from 1 to 14 October.

The ten most used: providers of each

  1. glm-5.3-flash36
  2. deepseek-v4.1-flash30
  3. deepseek-v4-flash-073126
  4. glm-5.224
  5. kimi-k321
  6. deepseek-v4-flash14
  7. mimo-v2.6-flash7
  8. hy4-preview7
  9. hy35
  10. nemotron-3-ultra-550b-a55b3

All open models

010203039 models with 1 providers39114 models with 2 providers14214 models with 3–4 providers1 of the ten most used143–414 models with 5–9 providers3 of the ten most used145–910 models with 10–19 providers1 of the ten most used1010–196 models with 20–29 providers3 of the ten most used620–292 models with 30 or more providers2 of the ten most used230 or moreProviders with an endpoint for the model on OpenRouterModels
010203039 models with 1 providers39114 models with 2 providers14214 models with 3–4 providers1 of the ten most used143–414 models with 5–9 providers3 of the ten most used145–910 models with 10–19 providers1 of the ten most used1010–196 models with 20–29 providers3 of the ten most used620–292 models with 30 or more providers2 of the ten most used230 or moreProviders per modelModels
A provider is a company with at least one endpoint for the model; the number of providers is not a market share and does not guarantee availability, price or the same precision. The ten most used are chosen by their tokens from 1 to 14 October 2026 and their providers are counted on 9 November. Source: OpenRouter, as displayed.

02 API or capacity

An API charges per token; dedicated GPUs charge per hour, busy or idle

That difference decides many cases. On their own price lists, many hosts of an open model charge what its lab does: 8 of the 9 that list Kimi K3 match Moonshot’s $3 and $15 per million tokens in and out. Renting GPUs gives control over the version, the precision and where requests are processed, but the hours are paid whether the GPUs work or not; owning them adds the hardware and the people who run it.

Three ways to run an open model

  1. 1

    A shared API

    a developer’s or a host’s endpoint

    You pay

    per token: Kimi K3 at $3.00 per million tokens in and $15.00 out, at Moonshot’s price

    Someone else decides

    the precision, the version, the rate limits and where requests are processed

    The cost depends on

    how many tokens you send

  2. 2

    Dedicated capacity

    GPUs a provider reserves for you

    You pay

    per GPU-hour, busy or idle: a B200 at a median $8.35 an hour on demand

    Someone else decides

    the hardware and the data centre; you choose the model, precision and serving setup

    The cost depends on

    how busy you keep the GPUs

  3. 3

    Self-managed infrastructure

    GPUs you own, or rent bare, and run yourself

    You pay

    for the GPUs (bought, or rented by the hour), power, space and the people who run them

    Someone else decides

    nothing: every choice and every failure is yours

    The cost depends on

    how busy you keep them, and for how long you keep them

Prices are examples from this chapter’s data: Moonshot’s own list price for Kimi K3 (9 November 2026) and the median public on-demand list price of a B200, collected on 26 October and 9 November 2026. The three can be combined. Sources: providers’ price pages.

How many hosts charge exactly what the developer charges, November 2026

At the developer’s own price At another price

Kimi K3 developer: $15.00 per M output tokens

8 of 91

GLM-5.3 developer: $4.40 per M output tokens

7 of 103

DeepSeek-V4-Pro developer: $3.96 per M output tokens

4 of 84

Qwen3.5-397B-A17B developer: $3.60 per M output tokens

3 of 41

Qwen3.8-27B developer: $3.00 per M output tokens

1 of 76

DeepSeek-V4.1-Flash developer: $1.20 per M output tokens

8 of 102

GLM-5.3-Flash developer: $0.50 per M output tokens

8 of 91
Hosts that list each model at a standard price on their own price page, 9 November 2026. On OpenRouter the spread is wider, because its cheapest endpoints usually declare 4-bit precision or run a promotion. Sources: providers’ price pages; OpenRouter, as displayed.

What tokens actually cost on one provider

Across the usage data Novita AI shared, the realized average price per million tokens fell 70.5% between June and October 2026, from $0.256 to $0.075. Tokens moved to cheaper models: the share on models with a realized price below $0.10 rose from 37.3% to 82.5%, while the average across models, each counted once, fell only 9.8%. Prices within that cheaper group fell as well, and the data do not separate how much of the drop each change explains. These are realized prices (revenue divided by tokens) on one provider, not list prices or a market-wide index.

Realized price per million tokens, per token and with each model counted once, and the share of tokens on models below $0.10, June–November 2026

$0.00$0.20$0.40$0.60JunJulAugSepOctNov*Each model once$0.387Per token$0.075

Share of tokens on models below $0.10 per million

  1. Jun37.3%
  2. Jul39.0%
  3. Aug59.8%
  4. Sep66.7%
  5. Oct82.5%
  6. Nov*69.9%
Realized price: revenue divided by tokens, input and output together. “Per token” weighs each model by its tokens; “each model once” is the plain average of the models served that month, a set that changes from month to month. *November is partial and not compared with full months. Source: Novita AI, aggregated usage and realized prices, June–November 2026 (preliminary contribution, November 2026; November partial; tencent/hy3 excluded).

None of the three is cheaper in general; each fits a different situation. A shared API suits traffic that is still unknown or comes in bursts: nothing is paid when nothing is sent. Dedicated capacity suits a steady load that keeps the GPUs busy, or a team that needs a fixed version and precision. Running one’s own infrastructure makes sense when the data must stay in-house and there is a team to operate it. The routes can be combined: an API for the peaks, capacity of one’s own for the base load.

Comparing prices hides one trap. Most hosts charge what the lab charges, and some of the cheaper offers on OpenRouter are cheaper because they run the model at lower precision: the cheapest endpoints of Kimi K3 and GLM-5.3 declare 4 bits. Before the price, read the precision.

03 When it pays

When a rented node pays off against the API depends on the assumptions: from 26% of the month to never

An example. Renting eight B200s to run DeepSeek-V4-Pro costs about $48,700 a month, whether they work or not. Paying DeepSeek’s API instead costs nothing when idle and a fixed price per token. The node is cheaper only if it is kept busy: in this example, working flat out for at least 56% of the month. Below that, the API is cheaper. Change one assumption, such as the GPU price, the length of the prompts or the API price compared against, and that share moves a lot.

How much of the month eight rented B200s must work to cost the same as DeepSeek’s API for DeepSeek-V4-Pro, under each assumption

  1. The chapter’s scenario56%
  2. GPUs at the spot price33%
  3. GPUs at the highest list price96%
  4. 10 input tokens per output token26%
  5. 1 input token per output token84%
  6. Single-turn throughput77%
  7. API at the cheapest OpenRouter pricenever

Novita AI’s observed mix, about 21 input tokens per output token14%

A modelled scenario, not an observed cost and not the total cost of production. Each row changes one assumption of the first; “never”: not even at full use. The measured throughput comes from a workload with far more input per output token than the API price assumes, so no row is an empirical threshold. Sources: providers’ price pages; SemiAnalysis InferenceX.
The same calculation for other models
  • Kimi K3 16 × B200not reached: needs 97 output tokens a second per GPU, measured 56
  • GLM-5.3 4 × B200no independent measurement on this hardware
  • Qwen3.5-397B-A17B 2 × B20068% of the month
  • DeepSeek-V4.1-Flash 8 × B20039% of the month
  • GLM-5.3-Flash 8 × B200not reached: needs 2,440 output tokens a second per GPU, measured 265

Per month, the rented node costs the same busy or idle; the API costs by the token

DeepSeek-V4-Pro on eight B200s at the median on-demand price, against what the API would charge for the tokens the node produces at its measured throughput, with three input tokens per output token.

$0$20,000$40,000$60,000$80,000$100,0000%25%50%75%100%Share of the month the node works at its measured throughputThe node is cheaper56%8 × B200 node: $48,735 a monthDeepSeek API: $7.92 per MCheapest endpoint: $3.96 per M
$0$20,000$40,000$60,000$80,000$100,0000%25%50%75%100%Share of the month the node worksThe node is cheaper56%Node: $48,735 a monthDeepSeek API: $7.92 per MCheapest endpoint: $3.96 per M
A modelled scenario, not an observed cost: rented hardware only, 730 hours a month. The measured throughput (SemiAnalysis InferenceX) comes from agentic traces with far more input per output token than the mix used to price the API, so the crossing is not a threshold. Sources: providers’ price pages; SemiAnalysis InferenceX.

The answer runs from 26% of the month to never. The share depends less on the model than on the assumptions: what the GPUs cost, how long the prompts are, how fast the model really runs and which API price it is compared with.

One observation moves the mix assumption. In the usage data Novita AI shared, input tokens were 95.4% of all tokens between June and November 2026, about 21 input tokens per output token, far from the three-to-one mix list prices are usually compared on. At that mix, the node in this example would cost the same as DeepSeek’s API at 14% of the month. It is one provider’s mix, and the measured throughput still comes from another workload.

04 The cost curve

The busier a rented node is kept, the cheaper each token; the API price does not move

A rented node costs the same every month. Spread over few tokens, each one is dear; over many, cheap. At full use, DeepSeek-V4-Pro on eight B200s produces a million output tokens for $4.43; DeepSeek’s own API charges $7.92 for the same mix, at any volume. The two lines cross at 56% of the month.

DeepSeek-V4-Pro on eight rented B200s: cost per million output tokens against the share of the month the node works

0%25%50%75%100%$0$10$20$30Dollars per million output tokensShare of the month the node runs at its measured throughputDeepSeek’s API: $7.92Cheapest OpenRouter endpoint: $3.96Rented node: 8 × B200The node is cheaper56%0%25%50%75%100%$0$10$20$30Dollars per million output tokensShare of the month at measured throughputDeepSeek’s API: $7.92Cheapest endpoint: $3.96Rented node: 8 × B200Node cheaper56%

Assumptions

  • The node is rented at the median public on-demand price, 730 hours a month.
  • The API is the developer’s own list price; each output token comes with three input tokens.
  • Throughput is an independent measurement (SemiAnalysis InferenceX) on agentic coding traces, with about 129 input tokens per output token, most of them cached. The two sides do not share a workload.
  • Only the rented hardware is counted: no staff, storage, network, redundancy, setup or idle capacity.
The node’s monthly cost divided by the tokens it produces at the measured throughput; the flat lines are DeepSeek’s own price and the cheapest OpenRouter endpoint for the same mix. Break-even utilisation = output tokens per second per GPU needed at full use ÷ measured output tokens per second per GPU. Sources: providers’ price pages; SemiAnalysis InferenceX.

Utilisation, the share of the month the GPUs are actually working, is the variable to watch: a node that sits idle at night or between bursts is paid for anyway. For two of the other models measured, the throughput is too low to reach the developer’s price at all. Owning capacity can pay for large and steady volumes, or when data must stay under the organisation’s control, but only a test on one’s own traffic can say where the line is.

05 Hardware and place

Memory, precision and place: what the price of a GPU hour does not say

Memory decides whether a model fits, bandwidth and compute how fast it runs, and the price what each hour costs. A B200 rents for nearly twice an H100, with more than twice its memory.

What a GPU rents for on demand: list prices of 26 October and 9 November 2026

Median price From the cheapest to the dearest list price

  1. B200 180 GB

    $8.35an hour

  2. H200 141 GB

    $6.15an hour

  3. H100 80 GB

    $4.46an hour

  4. RTX PRO 6000 96 GB

    $2.49an hour

Public on-demand list prices per GPU and hour. Spot and reserved capacity are cheaper. Sources: providers’ price pages; memory from the manufacturers’ datasheets.

For a European customer, place can weigh as much as price, and it is the hardest thing to check. Providers headquartered in the EU served 0.6% of the tokens on OpenRouter’s providers page, a share of one router’s traffic and not of inference in Europe. Alternatives exist: at least ten EU-based companies list this chapter’s Chinese open models in European data centres, mostly at or near the developer’s price, and their median on-demand H100 listed at $3.76 an hour. But a provider’s headquarters is not where it processes requests, and data residency is the provider’s own claim: ask where each request is processed.

06 Configurations

How many GPUs each open model needs, and what they cost a month

The largest open models need several GPUs, sometimes several servers. DeepSeek-V4-Pro is documented on eight H200s or eight B200s with its experts at 4 bits; Qwen3.5-397B-A17B on eight H200s at 8 bits or two B200s at 4 bits; only the dense Qwen3.8-27B fits on one card. Choose a model to see its servers and their monthly cost on demand.

Documented configurations: the GPUs that model makers and serving engines publish for each model, and their monthly cost on demand

Kimi K3 · 2.8 trillion parameters, 104 billion active

  1. 8 × B300

    $53,900a month on demand

    1 server · NVFP4 (third-party) · 2.3 TB of GPU memory

  2. 16 × B200

    $97,500a month on demand

    2 servers · NVFP4 (third-party) · 2.9 TB of GPU memory

Each server has eight GPU slots; the drawings show the GPUs one configuration needs. The configurations are the ones the model makers, the serving engines (vLLM, SGLang) and Nvidia document for each model. On Nvidia’s Blackwell GPUs (B200, B300 and RTX PRO 6000) they use NVFP4, Blackwell’s own four-bit format; “third-party” marks a conversion by someone other than the model’s developer. On H200 and H100 they keep the “developer’s release”: the number format in which the model’s developer publishes its weights. More GPUs give room for longer contexts and more users at once. Monthly cost: 730 hours at the median on-demand list price of each GPU. Sources: model cards, vLLM and SGLang recipes, Nvidia NIM support matrix; providers’ price pages.

07 Quantisation

The same model can be served at different precisions

Quantising stores a model’s weights in fewer bits: it needs less memory and runs faster, sometimes at a cost in quality. On OpenRouter, 43% of the endpoints of open models declare FP8 and 24% 4 bits; 25% declare nothing. Among the quantised copies people download from Hugging Face, GGUF leads, although the way the Hub counts its files inflates it.

The same model, three precisions: DeepSeek-V4-Pro’s 1.6 trillion weights in 16, 8 and 4 bits

PrecisionOne weightB200s needed just for the weightsBF1616 bits per weight3.2 TB18 B200FP88 bits per weight1.6 TB9 B2004-bit4 bits per weight0.8 TB5 B200
PrecisionB200s needed just for the weightsBF163.2 TB16 bits per weight18 B200FP81.6 TB8 bits per weight9 B2004-bit0.8 TB4 bits per weight5 B200
Weights only: parameters × bits ÷ 8, and how many 180 GB B200s that fills. A running model also needs memory for its context (the KV cache) and its activations, so real deployments use more GPUs; fewer bits also make each token faster to compute, sometimes at some cost in quality. Sources: DeepSeek-V4-Pro model card; memory from the manufacturer’s datasheet.

Precision changes between hosts and between downloads: two populations, measured separately

Endpoints: the precision each host declares, 595 endpoints of open models on OpenRouter

43% FP824% 4-bit (FP4, MXFP4, NVFP4, INT4)9% BF16 / FP16 / FP3225% Not declared

Downloads: the format of quantised copies on Hugging Face, share of 30-day downloads

55% GGUF11% MLX10% AWQ8% NVFP4 / FP47% FP88% Other
Top: the precision each endpoint declares, 595 endpoints of verified open-weight models, 9 November 2026 (declared, not verified). Bottom: 30-day downloads of quantised derivatives of the tracked open models, 22 October 2026; GGUF is inflated by how the Hub counts its files. Sources: OpenRouter, as displayed; Hugging Face hub-stats.

Two endpoints with the same model name can serve different precisions, and the price often follows the bits: the cheapest endpoints of Kimi K3 and GLM-5.3 on OpenRouter declare 4 bits. Before comparing two offers, check that they serve the same version at the precision the application needs.

Chapter 06 · Closing

How to decide where to run an open model

There is no single way to run a model whose weights are available. For a team that does not yet know its traffic, an API lets it start without committing to capacity. With heavy and predictable loads, a deployment of one’s own can pay, but the calculation depends on the throughput actually achieved, the price of the hardware and the work of running it. Before comparing two offers, check that they serve the same version at the precision the application needs, and ask where each request is processed.

Traffic still unknown
An API lets a team start without committing to capacity. Nothing is paid when nothing is sent.
Heavy and predictable load
A rented node run by the team itself can pay, but the calculation depends on the throughput actually achieved, the price of the hardware and the work of running it.
Before comparing two offers
Check that they serve the same version at the precision the application needs, and ask where each request is processed.