The distance depends on the task. On the tests this report uses, closed models are still ahead: by about two points in graduate-level science and in long agentic tasks, and by 17 to 32 points in advanced mathematics, scientific images and factual knowledge.
Sources: Epoch AI (CC BY 4.0); Vals AI; Artificial Analysis.
2–32points
of difference between the best open-weight model and the best closed model on four tests: 2.2 in graduate-level science, 16.8 in advanced mathematics, 31.8 in factual knowledge and 1.5 in long agentic tasks, on Epoch AI’s and Vals AI’s tests.
Epoch AI (CC BY 4.0); Vals AI.
Capability
In graduate-level science, the best open-weight model we can compare, Kimi K3.1 Pro, came within 2.2 points of the best closed one, Gemini 4 Argon. In long agentic tasks it came within 1.5. In advanced mathematics and factual knowledge the distance was 16.8 and 31.8 points. The distance changes from one task to another. But benchmarks are also starting to saturate, and we have changed the way we measure.
01 Four tests, four distances
The distance dependson the task
In graduate-level science, Kimi K3.1 Pro is the best open-weight model, 2.2 points behind Gemini 4 Argon. In factual knowledge, Claude Fable 5.2 is 31.8 points ahead of Kimi K3.1 Pro.
The best open-weight model and the best closed model on four tests
Best open-weight modelBest closed model
Graduate-level science
Kimi K3.1 Pro 94.2%
Gemini 4 Argon 96.4%
2.2 pts
Agentic work *
Kimi K3.1 Pro 63.6%
Claude Fable 5.2 and Claude Opus 5.5 65.2%
1.5 pts
Advanced mathematics
Kimi K3.1 Pro 78.2%
Claude Fable 5.2 95.1%
16.8 pts
Factual knowledge
Kimi K3.1 Pro 62.2%
Claude Fable 5.2 94.0%
31.8 pts
0%25%50%75%100%
Share of each test solved or answered correctly, models released by 10 November 2026, each at its best reasoning setting: Epoch AI’s own runs for three tests. *Agentic work: Vals AI’s Terminal-Bench 4.0, every model through the same agent, mean of three runs. Sources: Epoch AI (CC BY 4.0); Vals AI.5 more tests from Epoch AI
Competition mathematics (both at 100%)
Kimi K3.1 Pro 100.0%
Claude Fable 5.1 and 10 more 100.0%
0.0 pts
Software engineering (last run in June 2026)
GLM-5.2 78.7%
Claude Opus 4.7 83.5%
4.8 pts
Chess puzzles
DeepSeek V4.1 Flash 48.0%
Claude Fable 5.2 79.0%
31.0 pts
Research-level mathematics
Kimi K3.1 Pro 65.8%
Claude Fable 5.2 and GPT-6.1 Sol 100.0%
34.2 pts
Mystery puzzles
DeepSeek V4.1 Flash 49.0%
Claude Fable 5.2 90.0%
41.0 pts
0%25%50%75%100%
Same reading as the chart: models released by 10 November 2026, each at its best setting, in Epoch AI’s own runs; every other test in Epoch AI’s archive with an open-weight and a closed result. Epoch AI has not run SWE-bench Verified since June 2026, so newer models are missing from it. Source: Epoch AI (CC BY 4.0).
Two of the four distances are small: 2.2 points in science and 1.5 in long agentic work, both for Kimi K3.1 Pro. Two are large: 16.8 in advanced mathematics and 31.8 in answering factual questions from memory, where Claude Fable 5.2 is far ahead of every other model.
02 How far behind in time
How long open models taketo reach the closed models’ scores
Take the score the best open-weight model has today on each test, and look back: when did a closed model first reach it? In graduate-level science, Gemini 3.1 Pro Preview did in February 2026, 9 months earlier. Across the four tests, the answer runs from 2 to 11 months.
How many months earlier a closed model reached the score the best open-weight model has today
1
Factual knowledge Gemini 3 Flash Preview, December 2025
11months
2
Graduate-level science Gemini 3.1 Pro Preview, February 2026
9months
3
Advanced mathematics GPT-5.4 Pro, March 2026
8months
4
Agentic work Claude Opus 5.5, September 2026
2months
For each test: the score of the best open-weight model on 10 November 2026; the first month in which a closed model had reached it, and which model. Models released by each month’s end. Sources: Epoch AI (CC BY 4.0); Vals AI.The same reading on three more tests from Epoch AI
1
Chess puzzles GPT-5 Pro, October 2025
13months
2
Software engineering Claude Opus 4.6, February 2026
9months
3
Research-level mathematics GPT-5.5 Pro, April 2026
7months
4
Mystery puzzles GPT-5.5, April 2026
7months
5
Competition mathematics Claude Fable 5, June 2026
5months
For each test, the first month in which a closed model had reached the score the best open-weight model has today. Source: Epoch AI (CC BY 4.0).
Best score of an open-weight and of a closed model, by month, January 2024 to November 2026
Best open-weight modelBest closed modelBest open-weight score todayFirst month a closed model reached it
GPQA Diamond · 9 months
Terminal-Bench 4.0 · 2 months
FrontierMath Tiers 1-3 · 8 months
SimpleQA Verified · 11 months
The dashed line is the best open-weight score today; the dot, the first month a closed model reached it. Agentic work: Vals AI ran Terminal-Bench 4.0 on models released in 2026 only, so its lines start late. Sources: Epoch AI (CC BY 4.0); Vals AI.
The time it takes to reach the closed models is shortest where open models are closest: 2 months in agentic work and 9 in science. It is longest in factual knowledge, 11 months, where the gap in points is also the widest.
03 Saturation
In graduate-level science, the leading models areclose to the test’s ceiling
In graduate-level science, 30 of the 38 model versions released since July 2026 score above 90%. When almost every leading model is near the top, a difference of 2.2 points is not enough to base decisions on.
On GPQA Diamond the twelve best versions fit within 2.5 points and their standard errors overlap
Score of each version released since July 2026, with its standard error (±1) as Epoch AI publishes it. The axis starts at 85%: this is a dot chart, not a bar chart.
Open-weightClosed
When nearly every leading model scores above 90% and their margins overlap, their order says little. Source: Epoch AI, Benchmarking Hub (CC BY 4.0).
Kimi K3.1 Pro, the best open-weight model, is number 9 among these versions, in a group whose first and twelfth are about three points apart. On this test the ranking says little; the tests that still separate the leaders are the harder ones: advanced mathematics, factual recall, scientific images and long agentic tasks.
04 Scientific images
Reading laboratory images,the best open-weight model is 22 points behind
VIALS asks a model to interpret the images a life-sciences lab works with every day: gel blots, microscopy, flow cytometry, plasmid maps. Artificial Analysis runs every model three times on each of its 161 tasks and counts a task only when all three answers are right. Claude Fable 5.2 did so on 67.1% of the tasks; Kimi K3.1 Pro, the best open-weight model, on 45.3%.
VIALS-AA: share of scientific-image tasks answered correctly in all three attempts
Open-weightClosed
1
Claude Fable 5.2 Anthropic · #1 of 17
67.1%released Nov. 2026
2
GPT-6 Astra OpenAI · #2 of 17
62.7%released Sept. 2026
3
Claude Opus 5.5 Anthropic · #3 of 17
61.5%released Sept. 2026
4
GPT-6.1 Terra OpenAI · #4 of 17
60.2%released Oct. 2026
5
Claude Sonnet 5.5 Anthropic · #5 of 17
59.6%released Sept. 2026
6
Kimi K3.1 Pro Moonshot AI · #10 of 17
45.3%released Oct. 2026
7
DeepSeek V4.1 Flash 1017 DeepSeek · #12 of 17
40.4%released Oct. 2026
8
GLM-5.3 Flash Z.ai · #13 of 17
39.8%released Aug. 2026
1
Kimi K3.1 Pro Moonshot AI · #10 of 17
45.3%released Oct. 2026
2
DeepSeek V4.1 Flash 1017 DeepSeek · #12 of 17
40.4%released Oct. 2026
3
GLM-5.3 Flash Z.ai · #13 of 17
39.8%released Aug. 2026
4
Mistral Large 4 Mistral AI · #17 of 17
20.5%released Oct. 2026
161 tasks on 174 laboratory images (blots, microscopy, flow cytometry, plasmid maps, trees, structures). Each model answers every task three times with a Python sandbox and an image viewer; a task counts only if all three answers are right (pass^3). First tab: the five best models and the three best open-weight ones; second: every open-weight model, with its place among all 17. Models released by 10 November 2026. Source: Artificial Analysis, VIALS-AA, as displayed on 11 November 2026; benchmark by Handshake AI.
It is a new test, published on 6 November 2026. It shows a gap the science quiz hides: on GPQA Diamond Kimi K3.1 Pro is 2.2 points from the best closed model; reading the images scientists use, 21.7. Answering once is easier: on the average of the three attempts the two are 11.8 points apart.
05 Agentic work
In long-horizon agentic tasks,the best open-weight model is 1.5 points behind
Terminal-Bench 4.0 gives an agent hours to finish real work in a terminal: build a service, fix a system, train a model. Run by Vals AI with the same agent for every model, Claude Fable 5.2 and Claude Opus 5.5 completed 65.2% of the tasks and Kimi K3.1 Pro, the best open-weight model, 63.6%.
Terminal-Bench 4.0: share of tasks completed, the five best models, the three best open-weight ones and the best of DeepSeek, same agent for all
Open-weightClosed
1
Claude Fable 5.2 Anthropic · #1 of 47
65.2%released Nov. 2026
2
Claude Opus 5.5 Anthropic · #2 of 47
65.2%released Sept. 2026
3
Claude Sonnet 5.5 Anthropic · #3 of 47
64.1%released Sept. 2026
4
Kimi K3.1 Pro Moonshot AI · #4 of 47
63.6%released Oct. 2026
5
GPT-6 Astra OpenAI · #5 of 47
59.6%released Sept. 2026
6
GLM-5.3 Z.ai · #16 of 47
38.9%released Aug. 2026
7
GLM-5.3 Flash Z.ai · #18 of 47
37.4%released Aug. 2026
8
DeepSeek V4.1 Flash DeepSeek · #21 of 47
31.8%released Oct. 2026
Vals AI ran 47 models released by 10 November 2026 through the same agent (mini-SWE-agent) on the 66 tasks of Terminal-Bench 4.0, three times each; the share is the mean of the three runs, with no partial credit. A quarter of the tasks are software engineering; the rest are science, machine learning, operations, hardware, security and media, each with up to eight hours. The 18 best are listed in the Deep Dive. Vals AI ran no open-weight Qwen model; Qwen3.8 Max (#20) is closed. Source: Vals AI, Terminal-Bench 4.0, leaderboard of 9 November 2026.
Kimi K3.1 Pro beats GPT-6 Astra; the next open-weight model, GLM-5.3, completed 38.9%, 25 points lower. On the 18 software tasks alone the gap is 7.4 points, on too few tasks to read on its own. What each run cost and how long it took is in chapter 04.
Chapter 02 · Closing
How to usethese results
There is no single distance between open-weight and closed models. We like to think everything comes down to a benchmark score, but it depends a lot on what the model is for. On some tasks the measured gap is small; on others it is still large. Before choosing, check two things: whether the benchmark resembles the real work, and whether the models were evaluated in a comparable way. Then test the candidates on your own tasks.
What the model is for
There is no single distance between open-weight and closed models. We like to think everything comes down to a benchmark score, but it depends a lot on what the model is for.
Distance by task
Kimi K3.1 Pro is about two points from the best closed models in science and in long agentic work; in mathematics, scientific images and factual recall the gap is still 17 to 32 points.
Before choosing
Check two things: whether the benchmark resembles the real work, and whether the models were evaluated in a comparable way. Then test the candidates on your own tasks.