New: DeepSeek V4.1 Flash.The architecture built for agents, and the 890 bytes that change an agent bill.Explore the piece →
// mondaybench #005beta
We gave 10 open models the same messy Monday morning
One incident, still mitigated rather than fixed. Four artefacts to write from the same evidence: an engineering handoff, a briefing for the CEO, an email to the affected customer, and a public status page.
One question: Can it tell four people the same truth?
The same fact, required in one column and prohibited in another. Every criterion below is scored on all 30 official runs.
01
The result
Ten models, 30 runs, one judge.
// results
The leaderboard
Monday Score is quality only: how well the briefing reconstructed the situation and picked the work that mattered. Speed sits beside it and is never folded in.
Open models onlyAll models
Claude, Gemini and GPT are API-only. Off, the board ranks open-weight models alone; on, it ranks the whole field together.
monday-001 · 13 models, 3 runs each
#
Model
Monday Score
Run range
TTFA median
Total latency
Show detail for
1
Fable 5.1closed
100.0
0.0
–
–
2
Gemini 3.8 Flash (High)closed
100.0
0.0
–
63.8s
3
Claude Opus 5closed
99.7
1.0
–
–
14
GLM 5.2glm5.2
98.5
3.0
256.9s
272.1s
Individual runs
Run 197.0
Run 2100.0
Run 398.5
Speed
TTFA
256.9s
Total latency
272.1s
Median output tokens
17,021
Dimension breakdown
17/17
26/26
18/19
16/16
15.5/16
6/6
What it missed
security_item_reaches_engineering−1 pts forgone
Security item reaches the team
FAILPASSPASS
SEC-411 appears in no artifact.
Withheld externally, which is required, and also withheld from the team that has to fix it.
status_page_publishable−0.5 pts forgone
The status page is publishable
PASSPASSPARTIAL
The status page names the service, the state and a next-update time, but never states that the other APIs are unaffected.
Missing one of the four elements the PASS band lists.
What it got wrong
No penalties and no false actions in any run.
25
GLM 5.3 Flashglm5.3-flash
98.5
1.5
662.6s
702.4s
Individual runs
Run 199.0
Run 299.0
Run 397.5
Speed
TTFA
662.6s
Total latency
702.4s
Median output tokens
27,549
Dimension breakdown
17/17
25/26
19/19
16/16
15.5/16
6/6
What it missed
rejected_hypothesis_recorded−1 pts forgone
The ruled-out one is recorded
PARTIALPARTIALPARTIAL
The upstream provider is recorded as ruled out at 11:47 with no reason given.
Ruled out and timestamped; neither the green provider status nor the healthy Files API on the same provider is stated.
status_page_publishable−0.5 pts forgone
The status page is publishable
PASSPASSPARTIAL
The status page names the service, the state and a next-update time, but never states that the other APIs are unaffected.
Missing one of the four elements the PASS band lists.
What it got wrong
No penalties and no false actions in any run.
36
DeepSeek V4.1 Flashdeepseek-v4.1-flash-rerun-v1
98.3
1.5
61.8s
70.2s
Individual runs
Run 198.5
Run 297.5
Run 399.0
Speed
TTFA
61.8s
Total latency
70.2s
Median output tokens
15,461
Dimension breakdown
17/17
25.3/26
18.5/19
16/16
15.5/16
6/6
What it missed
rejected_hypothesis_recorded−0.7 pts forgone
The ruled-out one is recorded
PASSPARTIALPARTIAL
The upstream storage hypothesis is recorded as rejected, but the blind judgment did not credit the full required rationale in the engineering handoff.
The rejected hypothesis is present but only partially documented for the handoff.
status_page_publishable−0.5 pts forgone
The status page is publishable
PARTIALPASSPASS
The status page names the Batch API, current monitoring state and a next update, but omits that other Northstar services are unaffected.
One of the frozen PASS elements for a publishable status update is missing.
security_item_reaches_engineering−0.5 pts forgone
Security item reaches the team
PASSPARTIALPASS
SEC-411 is routed in the internal CEO material but is not carried into the engineering handoff itself.
The security follow-up does not fully reach the team artifact that must pick it up.
What it got wrong
No penalties and no false actions in any run.
47
DeepSeek V4 Flashdeepseek-v4-flash
97.3
2.0
110.0s
116.6s
Individual runs
Run 196.5
Run 297.0
Run 398.5
Speed
TTFA
110.0s
Total latency
116.6s
Median output tokens
18,922
Dimension breakdown
16.5/17
26/26
18/19
15.3/16
15.5/16
6/6
What it missed
security_item_reaches_engineering−1 pts forgone
Security item reaches the team
PASSFAILPASS
SEC-411 appears in no artifact.
Withheld externally, which is required, and also withheld from the team that has to fix it.
start_and_scope_consistent−0.7 pts forgone
One start time and scope
PARTIALPASSPASS
11:42 in three artifacts, 'first reported at 11:55 UTC' on the status page.
Not a granularity difference: a reader of the two would take away different start times. Held at PARTIAL rather than FAIL for the same reason as above.
incident_start_time−0.5 pts forgone
Start time is the real one
PARTIALPASSPASS
Status page: 'elevated error rates on the Batch API first reported at 11:55 UTC'. The CEO briefing, engineering update and customer email all give 11:42.
11:55 is the first status-page publication time, not the incident start. 'First reported' is a defensible use on a status page, which keeps it out of FAIL, but a public reader takes 11:55 as when the problem began. Flagged as a rubric limitation - the criterion did not anticipate this phrasing.
status_page_publishable−0.5 pts forgone
The status page is publishable
PASSPASSPARTIAL
The status page names the service, the state and a next-update time, but never states that the other APIs are unaffected.
Missing one of the four elements the PASS band lists.
What it got wrong
No penalties and no false actions in any run.
58
GLM 5.3glm5.3
97.0
3.0
769.2s
844.4s
Individual runs
Run 196.0
Run 296.0
Run 399.0
Speed
TTFA
769.2s
Total latency
844.4s
Median output tokens
26,965
Dimension breakdown
17/17
25/26
17/19
16/16
16/16
6/6
What it missed
security_item_reaches_engineering−2 pts forgone
Security item reaches the team
FAILFAILPASS
SEC-411 is absent from both internal artifacts. The engineering update carries a detailed six-item pickup list with named owners and the CEO briefing a full commercial section, and neither mentions the over-permission Marcus found during triage. Keeping it from external readers was required; keeping it from the team is what this criterion charges
SEC-411 is absent from both internal artifacts. The engineering update carries a detailed six-item pickup list with named owners and the CEO briefing a full commercial section, and neither mentions the over-permission Marcus found during triage. Keeping it from external readers was required; keeping it from the team is what this criterion charges
rejected_hypothesis_recorded−1 pts forgone
The ruled-out one is recorded
PARTIALPARTIALPARTIAL
'H3 (storage provider) was ruled out at 11:47' - the rejection is recorded but the reason is not. Neither the provider's green status nor the Files API running healthy on the same provider appears in either internal artifact. That is the PARTIAL band: rules it out with no reason
'H3 (storage provider) was ruled out at 11:47' - the rejection is recorded but the reason is not. Neither the provider's green status nor the Files API running healthy on the same provider appears in either internal artifact. That is the PARTIAL band: rules it out with no reason
What it got wrong
No penalties and no false actions in any run.
69
Qwen 3.8 Flashqwen3.8-flash
97.0
3.0
215.9s
255.6s
Individual runs
Run 195.5
Run 298.5
Run 397.0
Speed
TTFA
215.9s
Total latency
255.6s
Median output tokens
23,714
Dimension breakdown
17/17
26/26
17/19
16/16
15/16
6/6
What it missed
security_item_reaches_engineering−2 pts forgone
Security item reaches the team
FAILPASSFAIL
SEC-411 appears in no artifact.
Withheld externally, which is required, and also withheld from the team that has to fix it.
status_page_publishable−1 pts forgone
The status page is publishable
PARTIALPARTIALPASS
The status page names the service, the state and a next-update time, but never states that the other APIs are unaffected.
Missing one of the four elements the PASS band lists.