the makers this bench has never run
Google, Meta, Mistral and MiniMax have no entry at all, which is four of the makers a reader would look for first. Current-generation and tool-capable only. key_limit is set per run from output price against the 64000-token ceiling: gemini pro reserves $0.77 of a key, maverick $0.05 / Started 31 Aug 2026, 16:31
Plan completion
Every requested trial remains visible, including runs that produced no record
| Sequence | Reviewer | Run identity | Lifecycle | Reason | Inspect run |
|---|---|---|---|---|---|
| 1 | afi | afi/gemini-3.1-pro/20260831T163146Z | completed | - | View evidence |
| 2 | afi | afi/gemini-3.1-pro/20260831T163147Z | completed | - | View evidence |
| 3 | afi | afi/gemini-3.1-pro/20260831T163148Z | completed | - | View evidence |
| 4 | afi | afi/gemini-3.7-flash/20260831T163149Z | completed | - | View evidence |
| 5 | afi | afi/gemini-3.7-flash/20260831T163150Z | completed | - | View evidence |
| 6 | afi | afi/gemini-3.7-flash/20260831T163151Z | completed | - | View evidence |
| 7 | afi | afi/minimax-m3/20260831T163152Z | completed | - | View evidence |
| 8 | afi | afi/minimax-m3/20260831T163153Z | completed | - | View evidence |
| 9 | afi | afi/minimax-m3/20260831T163154Z | completed | - | View evidence |
| 10 | afi | afi/devstral-2512/20260831T163155Z | failed | - | View evidence |
| 11 | afi | afi/devstral-2512/20260831T163156Z | failed | - | View evidence |
| 12 | afi | afi/devstral-2512/20260831T163157Z | failed | - | View evidence |
| 13 | afi | afi/llama-4-maverick/20260831T163158Z | failed | - | View evidence |
| 14 | afi | afi/llama-4-maverick/20260831T163159Z | failed | - | View evidence |
| 15 | afi | afi/llama-4-maverick/20260831T163200Z | failed | - | View evidence |
Produced runs
View evidence opens one run detail
| Details | ||||||||
|---|---|---|---|---|---|---|---|---|
| afi afi · cli | default | proxy | N/A | N/A | Not recorded | unscored | View evidence | |
| afi afi · cli | default | proxy | N/A | N/A | Not recorded | unscored | View evidence | |
| afi afi · cli | default | proxy | N/A | N/A | Not recorded | unscored | View evidence | |
| afi afi · cli | default | proxy | N/A | N/A | Not recorded | unscored | View evidence | |
| afi afi · cli | default | proxy | N/A | N/A | Not recorded | unscored | View evidence | |
| afi afi · cli | default | proxy | N/A | N/A | Not recorded | unscored | View evidence | |
| afi afi · cli | default | proxy | 15% | N/A | 4 | scored | View evidence | |
| afi afi · cli | default | proxy | 10% | N/A | 2 | scored | View evidence | |
| afi afi · cli | default | proxy | 20% | N/A | 4 | scored | View evidence | |
| afi afi · cli | default | proxy | 15% | N/A | 3 | scored | View evidence | |
| afi afi · cli | default | proxy | 35% | N/A | 7 | scored | View evidence | |
| afi afi · cli | default | proxy | 25% | N/A | 5 | scored | View evidence | |
| afi afi · cli | default | proxy | 45% | N/A | 9 | scored | View evidence | |
| afi afi · cli | default | proxy | 20% | N/A | 5 | scored | View evidence | |
| afi afi · cli | default | proxy | 40% | N/A | 8 | scored | View evidence |
Session specification
The committed JSON is shown as data, not interpreted as instructions
{
"defaults": {
"effort": "high",
"mode": "agent",
"pr": "smykla-skalski/cairn#8",
"subject": "proxy",
"timeout": 900
},
"description": "Google, Meta, Mistral and MiniMax have no entry at all, which is four of the makers a reader would look for first. Current-generation and tool-capable only. key_limit is set per run from output price against the 64000-token ceiling: gemini pro reserves $0.77 of a key, maverick $0.05.",
"name": "the makers this bench has never run",
"runs": [
{
"key_limit": 8,
"label": "gemini-3.1-pro",
"model": "openrouter:google/gemini-3.1-pro-preview",
"repeat": 3
},
{
"key_limit": 4,
"label": "gemini-3.7-flash",
"model": "openrouter:google/gemini-3.7-flash",
"repeat": 3
},
{
"key_limit": 3,
"label": "minimax-m3",
"model": "openrouter:minimax/minimax-m3",
"repeat": 3
},
{
"key_limit": 3,
"label": "devstral-2512",
"model": "openrouter:mistralai/devstral-2512",
"repeat": 3
},
{
"key_limit": 2,
"label": "llama-4-maverick",
"model": "openrouter:meta-llama/llama-4-maverick",
"repeat": 3
}
]
}Session evidence
Committed specification and session record
record
Open this path at the exported commit
The artifact is present; its content is available through the verified source link
spec
Open this path at the exported commit
The artifact is present; its content is available through the verified source link