All tags
Topic: "benchmark-saturation"
not much happened today
claude-fable-5 mythos-5 gpt-5.5 claude-code fable-5 codex opus-4.8 kimi-k2.7-code anthropic artificial-analysis datacurve moonshot model-sovereignty export-controls coding-agent-evaluation benchmarking benchmark-gaming harness-quality benchmark-saturation open-source-models natolambert theo cohere kunchenguid clementdelangue dejavucoder ofirpress ramplabs
Anthropic suspended access to Claude Fable 5 and Mythos 5 due to US export controls, sparking a debate on model sovereignty and geopolitical risks for frontier AI vendors. Artificial Analysis updated its coding agent benchmark, replacing SWE-Bench Pro with DeepSWE, reshuffling rankings with Claude Code + Fable 5 [max] leading. Discussions highlighted the importance of harness quality versus pure model capability and concerns over benchmark saturation and realism. Additionally, Moonshot released the open-source model Kimi K2.7-Code.