// Chapter 04 · Cost

The Economics of Open Models

The price of a token is not enough to know what a task costs: on the same 66 long agentic tasks, the success rate changed the final cost, and the time changed the wait.

Data: Vals AI, Terminal-Bench 4.0, models released by 10 November 2026.

$7per solved task

with Kimi K3.1 Pro, open-weight, which completed 63.6% of the same 66 long agentic tasks; Claude Opus 5.5 completed 65.2% at $18.48.

Vals AI, Terminal-Bench 4.0.

Illustration: layered landscape with a pale sun
Price, speed, success

Model prices are quoted per million tokens, but a team pays for finished work, and the two do not always go together. On 66 long tasks run with the same agent, some of the lowest costs per solved task came from open-weight models, and so did some of the highest. At the top, one open-weight model completed almost as much as the leaders for less than half their cost.

47 models released by 10 November 2026, 14 with open weights and 33 closed, run by Vals AI with the same agent on Terminal-Bench 4.0, three times, at the serving provider’s prices. Long agentic tasks in a terminal, a quarter of them software engineering.

01 Cost per solved task

Three models completed about four tasks in ten; a solved task cost $3, $9 or $24

GLM-5.3 and GLM-5.3 Flash come from the same lab, Z.ai, and completed almost the same share of the tasks, 38.9% and 37.4%. A solved task cost 7 times as much with the larger model. Claude Haiku 5.5, closed, sits between them. Two things the price list leaves out decide the bill: how many tokens a model spends on each attempt, and how many attempts fail, because failures are paid for too.

The ten models that completed between 29% and 44% of the tasks: the tokens they spent and the cost of a solved task

Open-weight Closed The three in the headline

ModelTasks completedTokens per taskCost per task attemptedCost per solved taskMinutes per task
Muse EmberClosed · MetaTasks completed39.4%Tokens per task30.9 millionCost per task attempted$0.22Cost per solved task$0.55Minutes per task26
DeepSeek V4.1 FlashOpen-weight · DeepSeekTasks completed31.8%Tokens per task39.6 millionCost per task attempted$0.54Cost per solved task$1.70Minutes per task41
GLM-5.3 FlashOpen-weight · Z.aiTasks completed37.4%Tokens per task35.7 millionCost per task attempted$1.22Cost per solved task$3.26Minutes per task61
Claude Haiku 5.5Closed · AnthropicTasks completed36.4%Tokens per task45.5 millionCost per task attempted$3.12Cost per solved task$8.57Minutes per task52
GPT-6 SolClosed · OpenAITasks completed44.4%Tokens per task13.8 millionCost per task attempted$5.79Cost per solved task$13.03Minutes per task35
GPT-5.6 SolClosed · OpenAITasks completed37.9%Tokens per task10.5 millionCost per task attempted$7.98Cost per solved task$21.08Minutes per task34
GLM-5.3Open-weight · Z.aiTasks completed38.9%Tokens per task31.5 millionCost per task attempted$9.37Cost per solved task$24.11Minutes per task83
Qwen3.8 MaxClosed · AlibabaTasks completed34.3%Tokens per task33.9 millionCost per task attempted$10.63Cost per solved task$30.96Minutes per task148
Grok 4.7Closed · xAITasks completed28.8%Tokens per task18.2 millionCost per task attempted$18.01Cost per solved task$62.57Minutes per task50
Claude Fable 5Closed · AnthropicTasks completed41.4%Tokens per task19.8 millionCost per task attempted$30.34Cost per solved task$73.26Minutes per task68
Tokens per task: input and output, cached input included. Cost per task attempted: the API bill for one task at the serving provider’s prices, every step of the agent included, mean of three runs. Cost per solved task: cost per task attempted divided by the share of tasks completed, so failed attempts are paid for. Minutes: the agent’s wall-clock time per task. Rows from the cheapest solved task to the dearest. Source: Vals AI, Terminal-Bench 4.0, leaderboard of 9 November 2026.

The difference is in the bill per attempt, not in the tokens: GLM-5.3 spent $9.37 a task and GLM-5.3 Flash $1.22, with a similar number of tokens, because the larger model’s tokens cost more and the Flash version ran on a cheaper provider. Among the 39 models that completed at least 10% of the tasks, of the 741 pairs with different prices per token, in 197 the model with the cheaper token spent more per solved task.

02 Best value

Kimi K3.1 Pro completed almost as much as the leaders, at less than half their cost per solved task

Kimi K3.1 Pro, open-weight, completed 63.6% of the tasks at $7.18 per solved task. Claude Opus 5.5 and Claude Fable 5.2 completed 65.2% at $18.48 and $22.35. The cheapest strong result is closed, though: GPT-6.1 Sol completed 55.0% at $3.13.

Completing more tasks does not always cost more: success and cost per solved task, 47 models

Each dot is a model on the 66 tasks of Terminal-Bench 4.0, with the same agent and at the prices of the provider that served it. Further right, more tasks completed; lower, a cheaper solved task.

Open-weight Closed Frontier: no other model completes more tasks for less

Scatter chart of 46 models: share of tasks completed against cost per solved task. The values are in the table that follows.$0.25$1.00$2.50$10$25$100$250$10000%10%20%30%40%50%60%70%Tasks completed (mean of three runs)Cost per solved task (log scale)DeepSeek V4.1 Flash (Sept.) (open-weight): 19.7% of tasks, $2.54 per solved task, 48 min per taskGPT-6 Luna (closed): 13.6% of tasks, $2.60 per solved task, 33 min per taskDeepSeek V4 Flash (open-weight): 18.7% of tasks, $4.10 per solved task, 106 min per taskGPT-6.1 Terra (closed): 49.5% of tasks, $4.36 per solved task, 34 min per taskGPT-5.6 Luna (closed): 11.6% of tasks, $6.30 per solved task, 36 min per taskClaude Haiku 5.5 (closed): 36.4% of tasks, $8.57 per solved task, 52 min per taskMistral Large 4 (open release) (open-weight): 26.3% of tasks, $10.39 per solved task, 65 min per taskGPT-6 Sol (closed): 44.4% of tasks, $13.03 per solved task, 35 min per taskGPT-6 Astra (closed): 59.6% of tasks, $16.08 per solved task, 36 min per taskMuse Wonder Full (closed): 58.1% of tasks, $18.13 per solved task, 72 min per taskMistral Large 4 (open-weight): 27.8% of tasks, $18.65 per solved task, 100 min per taskGPT-5.6 Sol (closed): 37.9% of tasks, $21.08 per solved task, 34 min per taskClaude Fable 5.2 (closed): 65.2% of tasks, $22.35 per solved task, 55 min per taskDeepSeek V4 Pro (open-weight): 14.1% of tasks, $23.42 per solved task, 92 min per taskGPT-5.6 Terra (closed): 22.7% of tasks, $24.65 per solved task, 31 min per taskMuse Spark 1.3 Max (closed): 24.8% of tasks, $26.89 per solved task, 53 min per taskGrok 4.6 (closed): 17.2% of tasks, $29.48 per solved task, 29 min per taskClaude Fable 5.1 (closed): 58.1% of tasks, $29.57 per solved task, 60 min per taskMuse Wonder (closed): 49.5% of tasks, $29.61 per solved task, 55 min per taskGemini 4 Argon (closed): 57.6% of tasks, $30.64 per solved task, 54 min per taskQwen3.8 Max (closed): 34.3% of tasks, $30.96 per solved task, 148 min per taskDeepSeek V4 Pro (April) (open-weight): 11.1% of tasks, $31.07 per solved task, 86 min per taskClaude Opus 5 (closed): 53.5% of tasks, $34.74 per solved task, 67 min per taskGemini 3.8 Flash (closed): 19.7% of tasks, $43.90 per solved task, 107 min per taskGPT-5.4 mini (closed): 2.5% of tasks, $56.96 per solved task, 30 min per taskGrok 4.7 (closed): 28.8% of tasks, $62.57 per solved task, 50 min per taskGrok 4.5 (closed): 8.6% of tasks, $64.28 per solved task, 34 min per taskMuse Spark 1.3 (closed): 10.6% of tasks, $66.34 per solved task, 119 min per taskMuse Spark 1.2 (closed): 6.1% of tasks, $68.23 per solved task, 78 min per taskGemini 3.7 Flash (closed): 12.1% of tasks, $71.82 per solved task, 84 min per taskClaude Fable 5 (closed): 41.4% of tasks, $73.26 per solved task, 68 min per taskClaude Opus 4.8 (closed): 23.2% of tasks, $73.78 per solved task, 71 min per taskGemini 3.5 Flash (closed): 6.1% of tasks, $98.69 per solved task, 71 min per taskGemini 3.1 Pro Preview (closed): 2.5% of tasks, $162.85 per solved task, 19 min per taskClaude Sonnet 5 (closed): 10.6% of tasks, $245.62 per solved task, 105 min per taskInkling (open-weight): 0.5% of tasks, $494.26 per solved task, 23 min per taskMiniMax-M3 (open-weight): 1.0% of tasks, $715.79 per solved task, 76 min per taskMuse Ember (closed): 39.4% of tasks, $0.55 per solved task, 26 min per taskDeepSeek V4.1 Flash (open-weight): 31.8% of tasks, $1.70 per solved task, 41 min per taskGPT-6.1 Sol (closed): 55.0% of tasks, $3.13 per solved task, 45 min per taskGLM-5.3 Flash (open-weight): 37.4% of tasks, $3.26 per solved task, 61 min per taskKimi K3.1 Pro (open-weight): 63.6% of tasks, $7.18 per solved task, 49 min per taskClaude Sonnet 5.5 (closed): 64.1% of tasks, $16.76 per solved task, 82 min per taskClaude Opus 5.5 (closed): 65.2% of tasks, $18.48 per solved task, 64 min per taskGLM-5.3 (open-weight): 38.9% of tasks, $24.11 per solved task, 83 min per taskKimi K3 (open-weight): 17.2% of tasks, $72.67 per solved task, 189 min per taskMuse Ember · $0.55DeepSeek V4.1 Flash · $1.70GPT-6.1 Sol · $3.13GLM-5.3 Flash · $3.26Kimi K3.1 Pro · $7.18Claude Sonnet 5.5 · $16.76Claude Opus 5.5 · $18.48GLM-5.3 · $24.11Kimi K3 · $72.67
Scatter chart of 46 models: share of tasks completed against cost per solved task. The values are in the table that follows.$0.25$1.00$2.50$10$25$100$250$10000%20%40%60%Tasks completedCost per solved task (log)DeepSeek V4.1 Flash (Sept.) (open-weight): 19.7% of tasks, $2.54 per solved task, 48 min per taskGPT-6 Luna (closed): 13.6% of tasks, $2.60 per solved task, 33 min per taskDeepSeek V4 Flash (open-weight): 18.7% of tasks, $4.10 per solved task, 106 min per taskGPT-6.1 Terra (closed): 49.5% of tasks, $4.36 per solved task, 34 min per taskGPT-5.6 Luna (closed): 11.6% of tasks, $6.30 per solved task, 36 min per taskClaude Haiku 5.5 (closed): 36.4% of tasks, $8.57 per solved task, 52 min per taskMistral Large 4 (open release) (open-weight): 26.3% of tasks, $10.39 per solved task, 65 min per taskGPT-6 Sol (closed): 44.4% of tasks, $13.03 per solved task, 35 min per taskGPT-6 Astra (closed): 59.6% of tasks, $16.08 per solved task, 36 min per taskMuse Wonder Full (closed): 58.1% of tasks, $18.13 per solved task, 72 min per taskMistral Large 4 (open-weight): 27.8% of tasks, $18.65 per solved task, 100 min per taskGPT-5.6 Sol (closed): 37.9% of tasks, $21.08 per solved task, 34 min per taskClaude Fable 5.2 (closed): 65.2% of tasks, $22.35 per solved task, 55 min per taskDeepSeek V4 Pro (open-weight): 14.1% of tasks, $23.42 per solved task, 92 min per taskGPT-5.6 Terra (closed): 22.7% of tasks, $24.65 per solved task, 31 min per taskMuse Spark 1.3 Max (closed): 24.8% of tasks, $26.89 per solved task, 53 min per taskGrok 4.6 (closed): 17.2% of tasks, $29.48 per solved task, 29 min per taskClaude Fable 5.1 (closed): 58.1% of tasks, $29.57 per solved task, 60 min per taskMuse Wonder (closed): 49.5% of tasks, $29.61 per solved task, 55 min per taskGemini 4 Argon (closed): 57.6% of tasks, $30.64 per solved task, 54 min per taskQwen3.8 Max (closed): 34.3% of tasks, $30.96 per solved task, 148 min per taskDeepSeek V4 Pro (April) (open-weight): 11.1% of tasks, $31.07 per solved task, 86 min per taskClaude Opus 5 (closed): 53.5% of tasks, $34.74 per solved task, 67 min per taskGemini 3.8 Flash (closed): 19.7% of tasks, $43.90 per solved task, 107 min per taskGPT-5.4 mini (closed): 2.5% of tasks, $56.96 per solved task, 30 min per taskGrok 4.7 (closed): 28.8% of tasks, $62.57 per solved task, 50 min per taskGrok 4.5 (closed): 8.6% of tasks, $64.28 per solved task, 34 min per taskMuse Spark 1.3 (closed): 10.6% of tasks, $66.34 per solved task, 119 min per taskMuse Spark 1.2 (closed): 6.1% of tasks, $68.23 per solved task, 78 min per taskGemini 3.7 Flash (closed): 12.1% of tasks, $71.82 per solved task, 84 min per taskClaude Fable 5 (closed): 41.4% of tasks, $73.26 per solved task, 68 min per taskClaude Opus 4.8 (closed): 23.2% of tasks, $73.78 per solved task, 71 min per taskGemini 3.5 Flash (closed): 6.1% of tasks, $98.69 per solved task, 71 min per taskGemini 3.1 Pro Preview (closed): 2.5% of tasks, $162.85 per solved task, 19 min per taskClaude Sonnet 5 (closed): 10.6% of tasks, $245.62 per solved task, 105 min per taskInkling (open-weight): 0.5% of tasks, $494.26 per solved task, 23 min per taskMiniMax-M3 (open-weight): 1.0% of tasks, $715.79 per solved task, 76 min per taskMuse Ember (closed): 39.4% of tasks, $0.55 per solved task, 26 min per taskDeepSeek V4.1 Flash (open-weight): 31.8% of tasks, $1.70 per solved task, 41 min per taskGPT-6.1 Sol (closed): 55.0% of tasks, $3.13 per solved task, 45 min per taskGLM-5.3 Flash (open-weight): 37.4% of tasks, $3.26 per solved task, 61 min per taskKimi K3.1 Pro (open-weight): 63.6% of tasks, $7.18 per solved task, 49 min per taskClaude Sonnet 5.5 (closed): 64.1% of tasks, $16.76 per solved task, 82 min per taskClaude Opus 5.5 (closed): 65.2% of tasks, $18.48 per solved task, 64 min per taskGLM-5.3 (open-weight): 38.9% of tasks, $24.11 per solved task, 83 min per taskKimi K3 (open-weight): 17.2% of tasks, $72.67 per solved task, 189 min per taskMuse EmberDeepSeek V4.1 FlashGPT-6.1 SolGLM-5.3 FlashKimi K3.1 ProSonnet 5.5Opus 5.5GLM-5.3Kimi K3
The 13 open-weight models in the chart
ModelTasks completedCost per solved taskMinutes per taskOn the frontier
Kimi K3.1 Pro63.6%$7.1849yes
GLM-5.338.9%$24.1183
GLM-5.3 Flash37.4%$3.2661
DeepSeek V4.1 Flash31.8%$1.7041
Mistral Large 427.8%$18.65100
Mistral Large 4 (open release)26.3%$10.3965
DeepSeek V4.1 Flash (Sept.)19.7%$2.5448
DeepSeek V4 Flash18.7%$4.10106
Kimi K317.2%$72.67189
DeepSeek V4 Pro14.1%$23.4292
DeepSeek V4 Pro (April)11.1%$31.0786
MiniMax-M31.0%$715.7976
Inkling0.5%$494.2623
Pareto frontier computed on the published table: Claude Opus 5.5, Claude Sonnet 5.5, Kimi K3.1 Pro, GPT-6.1 Sol, Muse Ember. It counts only success and cost per solved task, not time, safety or availability. No dot: Inkling Small, which solved no task (it has no cost per solved task); that is why there are 46 dots for 47 models. Hover over a dot to see its values. Cost per solved task = what all the tasks cost, failures included, divided by the tasks completed. A single benchmark of long agentic tasks: not a general cost of each model. Source: Vals AI.

Cost per solved task: the 10 models that completed at least half of the tasks, and the open-weight models that completed at least a quarter

Open-weight Closed

The 10 models that completed at least half of the tasks

  1. 5

    GPT-6.1 Sol 55.0% completed · 45 min

    $3.13

  2. 10

    Kimi K3.1 Pro 63.6% completed · 49 min

    $7.18

  3. 14

    GPT-6 Astra 59.6% completed · 36 min

    $16.08

  4. 15

    Claude Sonnet 5.5 64.1% completed · 82 min

    $16.76

  5. 16

    Muse Wonder Full 58.1% completed · 72 min

    $18.13

  6. 17

    Claude Opus 5.5 65.2% completed · 64 min

    $18.48

  7. 20

    Claude Fable 5.2 65.2% completed · 55 min

    $22.35

  8. 26

    Claude Fable 5.1 58.1% completed · 60 min

    $29.57

  9. 28

    Gemini 4 Argon 57.6% completed · 54 min

    $30.64

  10. 31

    Claude Opus 5 53.5% completed · 67 min

    $34.74

The 5 open-weight models that completed between 25% and 50% of the tasks

  1. 2

    DeepSeek V4.1 Flash 31.8% completed · 41 min

    $1.70

  2. 6

    GLM-5.3 Flash 37.4% completed · 61 min

    $3.26

  3. 12

    Mistral Large 4 (open release) 26.3% completed · 65 min

    $10.39

  4. 18

    Mistral Large 4 27.8% completed · 100 min

    $18.65

  5. 22

    GLM-5.3 38.9% completed · 83 min

    $24.11

What the model spent on all the tasks divided by the tasks it completed, failed attempts included, at the serving provider’s prices. The number on the left is the model’s rank among all 46. Source: Vals AI, Terminal-Bench 4.0, leaderboard of 9 November 2026.

A model being open does not make a task cheap. In 18 of the 28 pairs of an open-weight and a closed model within three points of success, the open-weight model cost less per solved task, a median 2.6 times less; in the others it cost more. The pairs come from the models Vals AI ran and several share a model, so this is not a share of the market. Kimi K3, the previous Kimi, completed 17.2% at $72.67 per solved task, among the dearest of all. The cheapest result of all is Meta’s Muse Ember, closed, released the day before the cut-off: $0.55 per solved task at the price recorded in its run.

03 Speed

Closed models were usually faster: a median 54 minutes per task against 70

Time is where closed models lead. At the median, an open-weight model needed 70 minutes per task and a closed one 54. Among the strong models, GPT-6 Astra was the fastest, 36 minutes, and Kimi K3.1 Pro took 49, less than Claude Opus 5.5, which needed 64.

Minutes per task, the 10 models that completed at least half of the tasks

Open-weight Closed

  1. 1

    GPT-6 Astra $16.08 per solved task · 59.6% completed

    36 min

  2. 2

    GPT-6.1 Sol $3.13 per solved task · 55.0% completed

    45 min

  3. 3

    Kimi K3.1 Pro $7.18 per solved task · 63.6% completed

    49 min

  4. 4

    Gemini 4 Argon $30.64 per solved task · 57.6% completed

    54 min

  5. 5

    Claude Fable 5.2 $22.35 per solved task · 65.2% completed

    55 min

  6. 6

    Claude Fable 5.1 $29.57 per solved task · 58.1% completed

    60 min

  7. 7

    Claude Opus 5.5 $18.48 per solved task · 65.2% completed

    64 min

  8. 8

    Claude Opus 5 $34.74 per solved task · 53.5% completed

    67 min

  9. 9

    Muse Wonder Full $18.13 per solved task · 58.1% completed

    72 min

  10. 10

    Claude Sonnet 5.5 $16.76 per solved task · 64.1% completed

    82 min

Shorter is faster. Across the 47 models, the median was 70 minutes per task for the open-weight ones and 54 for the closed ones. Minutes are the agent’s wall-clock time per task, tools and tests included, with each model served by the provider named in the full table; another provider or one’s own hardware would change them. Source: Vals AI, Terminal-Bench 4.0, leaderboard of 9 November 2026.

A cheaper task is not always a slower one. Kimi K3.1 Pro was cheaper and faster than Claude Opus 5.5, Claude Fable 5.2 and Claude Sonnet 5.5. GPT-6.1 Sol was cheaper and faster still, but completed 9 points fewer tasks. When the work runs in the background, overnight or in parallel, minutes count for less; when a developer is waiting on the agent, they are the product.

Chapter 04 · Closing

What to measure before choosing

The price per token is not the cost of a result. What a team pays for is each solved task, and that depends on the tokens a model spends on each attempt and on the attempts that fail, which are paid for too. In this benchmark some open-weight models solved tasks for a fraction of what closed models with similar success cost, others cost more, and the closed models were usually faster: there is no general advantage of open models, but particular models that compete on cost. The choice depends on the quality the task needs and on how much time matters. It is a single benchmark of long agentic tasks: before choosing, measure cost per solved task, success rate and time on tasks that resemble your own.

Price per token
The price per token is not the cost of a result. Among the 39 models that completed at least 10% of the tasks, of the 741 pairs with different prices per token, in 197 the model with the cheaper token spent more per solved task.
Open-weight and closed models
In this benchmark some open-weight models solved tasks for a fraction of what closed models with similar success cost, others cost more, and the closed models were usually faster.
A single benchmark
It is a single benchmark of long agentic tasks: before choosing, measure cost per solved task, success rate and time on tasks that resemble your own.