Chapter 5: What open-weight models actually change

Overview#

When someone suggests replacing a proprietary model with Qwen, Gemma, DeepSeek or GLM, the conversation often moves straight to GPUs. But choosing an open-weight model doesn’t mean you have to run it yourself.

You can use a metered API, buy dedicated capacity from a provider, or operate the weights on your own infrastructure where the license permits it. This matters for cost, because the organization that develops the model doesn’t have to be the one serving every request.

Open weights are free to download, not free to run. Each of those options comes with an inference bill, paid in tokens, committed capacity or GPUs. Nor is an open-weight model necessarily “open source” in the broader sense. Publishing the weights doesn’t imply publishing training data, a reproducible training pipeline or unrestricted use rights.

Practical#

First establish that the specific model meets the task requirements, then compare ways to deploy it.

Review licensing per model release, not per family: Qwen, Gemma, DeepSeek and GLM have each issued releases with different terms or dependencies. A repository’s code license may not cover the weights, and distilled variants may inherit conditions from their base models. Check the exact model card and license text before deploying commercially.

Weights developed in one country can run on infrastructure operated elsewhere, so a model’s origin tells you little about where your data is processed. For data handling, look at who operates the service, the region, logging, external tools and the contract. An OpenAI-compatible API doesn’t guarantee the same data path or feature behavior across providers.

Ledgerline. For extraction, Ledgerline’s shortlist included an open-weight model behind a managed metered API in the EU. Legal read the license of that exact release, the security team checked where the provider processes data, and the evaluation set from chapter 3 decided between it and the proprietary small model. Nobody bought a GPU.

Deep Dive#

Parameter counts map poorly to serving cost; GPU inference and deployment economics covers memory sizing for dense and mixture-of-experts models.

Two services exposing the same weight release can still differ in quantization, tokenization or chat templates, tool parsers, reasoning configuration, throughput and availability. Changing a base URL may reduce migration work, but you still have to validate and operate the new setup.

The cost of switching later#

Open weights and OpenAI-compatible APIs make it easier to leave a provider that raises prices, retires a model or changes terms. The move still takes work:

• Re-run your evaluation set against the new setup. Choose the model the task needs covers building one.
• Adjust prompts and tool schemas when the new service uses a different chat template, tool parser or structured-output support.
• Re-embed stored documents if the embedding model changes, because vectors from different models can’t be compared with each other.
• Export your data, logs and any fine-tuned weights, and budget for egress charges on the transfer.
• Read the contract. Minimum commitments, prepaid credits and notice periods can make leaving expensive before any technical work starts.

You can lower this cost from the start. Keep prompts, the evaluation set and request logging in your own systems, in formats that don’t depend on one provider. Keep a short record of the features you use that only one provider offers, such as a particular caching mode or batch interface, so a migration plan knows what it has to replace.

Official model-release references: Qwen3, Gemma documentation, DeepSeek-R1, DeepSeek-V3, GLM-4.5, GLM-5. Each release carries its own license terms.