The price of a token is not enough to know what a task costs: on the same 66 long agentic tasks, the success rate changed the final cost, and the time changed the wait.
Data: Vals AI, Terminal-Bench 4.0, models released by 10 November 2026.
$7per solved task
with Kimi K3.1 Pro, open-weight, which completed 63.6% of the same 66 long agentic tasks; Claude Opus 5.5 completed 65.2% at $18.48.
Vals AI, Terminal-Bench 4.0.
Price, speed, success
Model prices are quoted per million tokens, but a team pays for finished work, and the two do not always go together. On 66 long tasks run with the same agent, some of the lowest costs per solved task came from open-weight models, and so did some of the highest. At the top, one open-weight model completed almost as much as the leaders for less than half their cost.
47 models released by 10 November 2026, 14 with open weights and 33 closed, run by Vals AI with the same agent on Terminal-Bench 4.0, three times, at the serving provider’s prices. Long agentic tasks in a terminal, a quarter of them software engineering.
01 Cost per solved task
Three models completed about four tasks in ten;a solved task cost $3, $9 or $24
GLM-5.3 and GLM-5.3 Flash come from the same lab, Z.ai, and completed almost the same share of the tasks, 38.9% and 37.4%. A solved task cost 7 times as much with the larger model. Claude Haiku 5.5, closed, sits between them. Two things the price list leaves out decide the bill: how many tokens a model spends on each attempt, and how many attempts fail, because failures are paid for too.
The ten models that completed between 29% and 44% of the tasks: the tokens they spent and the cost of a solved task
Open-weightClosedThe three in the headline
Model
Tasks completed
Tokens per task
Cost per task attempted
Cost per solved task
Minutes per task
Muse EmberClosed · Meta
Tasks completed39.4%
Tokens per task30.9 million
Cost per task attempted$0.22
Cost per solved task$0.55
Minutes per task26
DeepSeek V4.1 FlashOpen-weight · DeepSeek
Tasks completed31.8%
Tokens per task39.6 million
Cost per task attempted$0.54
Cost per solved task$1.70
Minutes per task41
GLM-5.3 FlashOpen-weight · Z.ai
Tasks completed37.4%
Tokens per task35.7 million
Cost per task attempted$1.22
Cost per solved task$3.26
Minutes per task61
Claude Haiku 5.5Closed · Anthropic
Tasks completed36.4%
Tokens per task45.5 million
Cost per task attempted$3.12
Cost per solved task$8.57
Minutes per task52
GPT-6 SolClosed · OpenAI
Tasks completed44.4%
Tokens per task13.8 million
Cost per task attempted$5.79
Cost per solved task$13.03
Minutes per task35
GPT-5.6 SolClosed · OpenAI
Tasks completed37.9%
Tokens per task10.5 million
Cost per task attempted$7.98
Cost per solved task$21.08
Minutes per task34
GLM-5.3Open-weight · Z.ai
Tasks completed38.9%
Tokens per task31.5 million
Cost per task attempted$9.37
Cost per solved task$24.11
Minutes per task83
Qwen3.8 MaxClosed · Alibaba
Tasks completed34.3%
Tokens per task33.9 million
Cost per task attempted$10.63
Cost per solved task$30.96
Minutes per task148
Grok 4.7Closed · xAI
Tasks completed28.8%
Tokens per task18.2 million
Cost per task attempted$18.01
Cost per solved task$62.57
Minutes per task50
Claude Fable 5Closed · Anthropic
Tasks completed41.4%
Tokens per task19.8 million
Cost per task attempted$30.34
Cost per solved task$73.26
Minutes per task68
Tokens per task: input and output, cached input included. Cost per task attempted: the API bill for one task at the serving provider’s prices, every step of the agent included, mean of three runs. Cost per solved task: cost per task attempted divided by the share of tasks completed, so failed attempts are paid for. Minutes: the agent’s wall-clock time per task. Rows from the cheapest solved task to the dearest. Source: Vals AI, Terminal-Bench 4.0, leaderboard of 9 November 2026.
The difference is in the bill per attempt, not in the tokens: GLM-5.3 spent $9.37 a task and GLM-5.3 Flash $1.22, with a similar number of tokens, because the larger model’s tokens cost more and the Flash version ran on a cheaper provider. Among the 39 models that completed at least 10% of the tasks, of the 741 pairs with different prices per token, in 197 the model with the cheaper token spent more per solved task.
02 Best value
Kimi K3.1 Pro completed almost as much as the leaders,at less than half their cost per solved task
Kimi K3.1 Pro, open-weight, completed 63.6% of the tasks at $7.18 per solved task. Claude Opus 5.5 and Claude Fable 5.2 completed 65.2% at $18.48 and $22.35. The cheapest strong result is closed, though: GPT-6.1 Sol completed 55.0% at $3.13.
Completing more tasks does not always cost more: success and cost per solved task, 47 models
Each dot is a model on the 66 tasks of Terminal-Bench 4.0, with the same agent and at the prices of the provider that served it. Further right, more tasks completed; lower, a cheaper solved task.
Open-weightClosedFrontier: no other model completes more tasks for less
The 13 open-weight models in the chart
Model
Tasks completed
Cost per solved task
Minutes per task
On the frontier
Kimi K3.1 Pro
63.6%
$7.18
49
yes
GLM-5.3
38.9%
$24.11
83
GLM-5.3 Flash
37.4%
$3.26
61
DeepSeek V4.1 Flash
31.8%
$1.70
41
Mistral Large 4
27.8%
$18.65
100
Mistral Large 4 (open release)
26.3%
$10.39
65
DeepSeek V4.1 Flash (Sept.)
19.7%
$2.54
48
DeepSeek V4 Flash
18.7%
$4.10
106
Kimi K3
17.2%
$72.67
189
DeepSeek V4 Pro
14.1%
$23.42
92
DeepSeek V4 Pro (April)
11.1%
$31.07
86
MiniMax-M3
1.0%
$715.79
76
Inkling
0.5%
$494.26
23
Pareto frontier computed on the published table: Claude Opus 5.5, Claude Sonnet 5.5, Kimi K3.1 Pro, GPT-6.1 Sol, Muse Ember. It counts only success and cost per solved task, not time, safety or availability. No dot: Inkling Small, which solved no task (it has no cost per solved task); that is why there are 46 dots for 47 models. Hover over a dot to see its values. Cost per solved task = what all the tasks cost, failures included, divided by the tasks completed. A single benchmark of long agentic tasks: not a general cost of each model. Source: Vals AI.
Cost per solved task: the 10 models that completed at least half of the tasks, and the open-weight models that completed at least a quarter
Open-weightClosed
The 10 models that completed at least half of the tasks
5
GPT-6.1 Sol 55.0% completed · 45 min
$3.13
10
Kimi K3.1 Pro 63.6% completed · 49 min
$7.18
14
GPT-6 Astra 59.6% completed · 36 min
$16.08
15
Claude Sonnet 5.5 64.1% completed · 82 min
$16.76
16
Muse Wonder Full 58.1% completed · 72 min
$18.13
17
Claude Opus 5.5 65.2% completed · 64 min
$18.48
20
Claude Fable 5.2 65.2% completed · 55 min
$22.35
26
Claude Fable 5.1 58.1% completed · 60 min
$29.57
28
Gemini 4 Argon 57.6% completed · 54 min
$30.64
31
Claude Opus 5 53.5% completed · 67 min
$34.74
The 5 open-weight models that completed between 25% and 50% of the tasks
2
DeepSeek V4.1 Flash 31.8% completed · 41 min
$1.70
6
GLM-5.3 Flash 37.4% completed · 61 min
$3.26
12
Mistral Large 4 (open release) 26.3% completed · 65 min
$10.39
18
Mistral Large 4 27.8% completed · 100 min
$18.65
22
GLM-5.3 38.9% completed · 83 min
$24.11
What the model spent on all the tasks divided by the tasks it completed, failed attempts included, at the serving provider’s prices. The number on the left is the model’s rank among all 46. Source: Vals AI, Terminal-Bench 4.0, leaderboard of 9 November 2026.
A model being open does not make a task cheap. In 18 of the 28 pairs of an open-weight and a closed model within three points of success, the open-weight model cost less per solved task, a median 2.6 times less; in the others it cost more. The pairs come from the models Vals AI ran and several share a model, so this is not a share of the market. Kimi K3, the previous Kimi, completed 17.2% at $72.67 per solved task, among the dearest of all. The cheapest result of all is Meta’s Muse Ember, closed, released the day before the cut-off: $0.55 per solved task at the price recorded in its run.
03 Speed
Closed models were usually faster:a median 54 minutes per task against 70
Time is where closed models lead. At the median, an open-weight model needed 70 minutes per task and a closed one 54. Among the strong models, GPT-6 Astra was the fastest, 36 minutes, and Kimi K3.1 Pro took 49, less than Claude Opus 5.5, which needed 64.
Minutes per task, the 10 models that completed at least half of the tasks
Open-weightClosed
1
GPT-6 Astra $16.08 per solved task · 59.6% completed
36 min
2
GPT-6.1 Sol $3.13 per solved task · 55.0% completed
45 min
3
Kimi K3.1 Pro $7.18 per solved task · 63.6% completed
49 min
4
Gemini 4 Argon $30.64 per solved task · 57.6% completed
54 min
5
Claude Fable 5.2 $22.35 per solved task · 65.2% completed
55 min
6
Claude Fable 5.1 $29.57 per solved task · 58.1% completed
60 min
7
Claude Opus 5.5 $18.48 per solved task · 65.2% completed
64 min
8
Claude Opus 5 $34.74 per solved task · 53.5% completed
67 min
9
Muse Wonder Full $18.13 per solved task · 58.1% completed
72 min
10
Claude Sonnet 5.5 $16.76 per solved task · 64.1% completed
82 min
Shorter is faster. Across the 47 models, the median was 70 minutes per task for the open-weight ones and 54 for the closed ones. Minutes are the agent’s wall-clock time per task, tools and tests included, with each model served by the provider named in the full table; another provider or one’s own hardware would change them. Source: Vals AI, Terminal-Bench 4.0, leaderboard of 9 November 2026.
A cheaper task is not always a slower one. Kimi K3.1 Pro was cheaper and faster than Claude Opus 5.5, Claude Fable 5.2 and Claude Sonnet 5.5. GPT-6.1 Sol was cheaper and faster still, but completed 9 points fewer tasks. When the work runs in the background, overnight or in parallel, minutes count for less; when a developer is waiting on the agent, they are the product.
Chapter 04 · Closing
What to measurebefore choosing
The price per token is not the cost of a result. What a team pays for is each solved task, and that depends on the tokens a model spends on each attempt and on the attempts that fail, which are paid for too. In this benchmark some open-weight models solved tasks for a fraction of what closed models with similar success cost, others cost more, and the closed models were usually faster: there is no general advantage of open models, but particular models that compete on cost. The choice depends on the quality the task needs and on how much time matters. It is a single benchmark of long agentic tasks: before choosing, measure cost per solved task, success rate and time on tasks that resemble your own.
Price per token
The price per token is not the cost of a result. Among the 39 models that completed at least 10% of the tasks, of the 741 pairs with different prices per token, in 197 the model with the cheaper token spent more per solved task.
Open-weight and closed models
In this benchmark some open-weight models solved tasks for a fraction of what closed models with similar success cost, others cost more, and the closed models were usually faster.
A single benchmark
It is a single benchmark of long agentic tasks: before choosing, measure cost per solved task, success rate and time on tasks that resemble your own.