We tested an architecture with rules and Clef, Cloudflare's decision model. DeepSeek went from 83% to 100% success and beat Claude Opus 5.5.
What we tested and why
We work with open models every day, and we know where they are good, where they are not, and where they can end up better than closed models.
So we asked our friends at Agent Swarm to help us run a series of experiments on how to improve open models when they work as agents.
Agent Swarm is open source software that provides an architecture in which one model acts as the coordinator or lead agent (the *Lead*): it breaks the task into subtasks and hands them out to several specialised subagents, the *workers*, which run them in parallel.
We wanted to know how good DeepSeek is at coordinating and running a "swarm" of agents. So Eze and Taras , from Agent Swarm, tested a coordination architecture that adds Clef as a decision model, together with a few explicit rules.
In the benchmark, DeepSeek got 28 out of 28 right and Claude Opus 5.5 got 28 out of 30. A 100% success rate that deserves a closer look and a proper explanation of the experiment. You can see all the benchmark results .
Inside the experiment
Without Clef or rules, DeepSeek had reached 83%. With the changes, it reached 100%.
What did we change? We did not touch the model. We changed how the system around it makes decisions.
We think this is a very important finding for today's open model ecosystem. The question we asked ourselves was: how far can we get better results from open models by optimising their architecture, without always relying on a frontier model as coordinator?
Running and coordinating are not the same
A coding agent gets a task, explores the repository, edits files, runs the tests and hands over the result. Many of us already do this every day.
But what happens when you bring in several agents?
When you put a swarm of agents to work, they need to coordinate so they do not repeat the same tasks or edit files someone else is touching. That is where a key role appears: the coordinator agent. An agent that does not need to solve problems, but to decide what to delegate, to whom, when to step in and when a task is done.
One bad decision can make the team repeat work, get stuck or deliver something that is not right.
Our hypothesis is that these decisions are orthogonal to the general reasoning an LLM provides.
Extensions as guardrails
The strategy here is simple. Systems like Agent Swarm let you build your own controllers for your organisation, called extensions. They run even before your favourite harness starts, they are independent, and they can be as deterministic as you want.
With this setup you can add extensions that validate your agent's input or output. In this case, to evaluate or classify a decision. In others, to sanitise what the agent has seen on the web, protect your users' data or swap tools without the agent noticing.
The architecture: what a "System One" model like Clef adds
If you have heard of Jev, which caused a small revolution a few months ago, you may know Clef. It is Cloudflare's decision model, and it is open source.
What matters is that it works differently from a generative LLM:
- It receives information and a set of possible answers (categories).
- It returns three values: the category, its probability and a yes/no.
- It works with less context: Jev, for example, has a 32,000-token input limit.
- It is very fast: around 200 ms per answer.
- It is pseudo-deterministic: for the same input, it always returns the same output.
The last point is key. With structured inputs that repeat, you can learn which cases it gets wrong, even when its confidence is high.
An example
Picture a coordinator agent with three workers. One finishes its task, another is still running and the third is stuck.
The system has to decide whether to wait, step in or reassign the work. Three possible answers: a simple classification. That is the kind of structured decision where Clef shines.
The architecture we tested
Agent Swarm added explicit rules to make sure the Lead delegated clearly, to the right tools and in a valid way. That way Clef gave fast, direct feedback to the coordination system.
- Agent Swarm organises the work.
- DeepSeek acts as the main model.
- The rules and Clef oversee certain decisions.
- The workers run the tasks.
- Helmcode serves the inference for the open models.
Every time an agent proposes an action, the gate summarises the context in roughly 1,000 tokens at most and Clef answers three questions. If all come out low, the action runs. If any comes out high, it is blocked and the agent gets the reason.
The Lead also has a fixed rule: it cannot read the task records itself. If it tries, the read is blocked and it has to delegate it to a worker. Any other read goes through Clef, which decides whether it makes sense for the Lead to do it at that moment.
What changed
The problem with today's open weight models is that they do not like being the boss. They like doing. That bias towards solving puts them at a clear disadvantage in complex systems: their first instinct is to answer the question, not to understand the complexity of the task and find the best agents to carry it through.
Clef forces them to rethink their actions with precise, constant guidance. That feedback loop, fast and cheap, fundamentally changes how the work gets done.
The results
| Model | Initial (29 Oct) | Rule + Clef | Claude Opus 5.5 |
|---|---|---|---|
| GLM 5.3 | 53% (9/17) | 70% (21/30) | 93% (28/30) |
| GLM 5.3 Flash | 35% (6/17) | 47% (14/30) | 93% (28/30) |
| Qwen 3.6 | 56% (10/18) | 53% (16/30) | 93% (28/30) |
| Gemma 4 | 88% (14/16) | 72% (18/25) | 93% (28/30) |
| DeepSeek | 83% (15/18) | 100% (28/28) | 93% (28/30) |
Source: [Agent Swarm × Helmcode benchmark](https://api.desplega.agent-swarm.dev/p/ab1bf515eb3a4f819d725fe29ac2c058), 10 November 2026.
DeepSeek's result
DeepSeek goes from 83% to 100%.
On its own, DeepSeek is not as good a coordinator as it is with Clef and the rules, which lift it to 100% success in every run of the experiment.
The most important part is the comparison with the other models and with Opus 5.5, which stays at 93% with two failed runs.
The other models
Not all of them improve. GLM 5.3 and GLM 5.3 Flash improve substantially, while Qwen 3.6 and Gemma 4 get worse.
This matters: the architecture does not work the same way for every model. Perhaps models with less reasoning capacity do not make better decisions with a decision model on top.
Methodology
These numbers should be read with care:
- The samples are small, and the number of attempts varies between models and configurations (from 16 to 30).
- DeepSeek's 28 out of 28 with rule + Clef comes from a 14-task run.
- The harnesses are not identical: Opus 5.5 ran on Claude Code (25 October) and the open models on OpenCode.
- Attempts lost to Helmcode infrastructure errors (HTTP 400 and 429), which spiked in some runs, are excluded.
The full data is on the benchmark page .
Findings we think matter
The model does not work alone
When we evaluate models we usually look at their capabilities, their benchmarks and their price. But an agent also depends on the harness, the tools, how it manages context and the decisions the system makes.
This experiment is a good example of how those elements can change the outcome.
Not every decision has to go through a general-purpose LLM
Agent Swarm has published several studies on this: on how much automating decisions saves and on how their agents "dream" every night . The key is to understand that an agentic system should look for determinism: to gain speed, cost and quality, you need the right tool for each job. In this case, code and "System One" models.
The system has to learn which decisions can be solved with rules, which with decision models and which really need more complex reasoning. It is not trivial: it means constantly examining the system's activity to spot those opportunities.
What this means for open models
What you see on day one can be very far from the results a week later.
There is no denying that almost every open model is a step (or several) behind the closed ones. That is why improving their reliability through architecture, and even more so with other open models like Clef, can make them competitive in more tasks without having to keep growing the model.
Next steps
We are going to build a more complex system on these same principles, to see whether these results hold up in scenarios far more demanding than our benchmark.
What would happen if we put this same architecture to work reviewing PRs, in a software factory, handling on-call incidents or doing security reviews?
The goal will be to check whether the results hold and to compare more equivalent configurations.
Thanks to Eze and Taras for designing and running the experiment. You can find out more about Agent Swarm and see the full benchmark .