It’s a good thing we published our Opus 5.5 podcast calling it the world’s best model before today’s launch.

In a very pleasant surprise, Gemini announced Carbon, a newer frontier model than the 3 week old Argon, with fantastic benchmarks:

Taking it to the new #1 on the Artificial Analysis consolidated benchmark, putting GDM in the lead for possibly the first time in AAII history:

sd

AI Twitter Recap

Gemini 4 Carbon Tops Independent Leaderboards, with Access Limited to Cyber Defenders

• Launch: Google DeepMind announced Gemini 4 Carbon two weeks after Argon. It is positioned as state of the art on cybersecurity and long-horizon tasks (@sundarpichai, @GoogleDeepMind).

  • Claimed cyber results: 88.2% on SEC-bench Pro for vulnerability finding, 54.8% on ExploitGym for exploitation and 71.0% on CWE-bench for patching (@sundarpichai).

  • Availability: Carbon first goes to trusted cyber defenders through the Fairwind Program, with wider developer access promised "as soon as possible" (@GoogleAI).

  • Pricing: Phil Schmid quotes $2/$10 per million input/output tokens with an asterisk (@_philschmid). Artificial Analysis lists $4/$20 standard pricing with a 50% launch discount that has no end date. Cached input costs $0.20 per million tokens (@ArtificialAnlys).

  • Specs: The model has a 1M-token context window and 1M max output. Inputs are text, image, video and speech; output is text (@ValsAI).

• Safety claims (Google): Successful cheating on Terminal-Bench 4.0 fell to 0.0%, and cheating attempts fell to 0.4%. Unsupported claims are down about 2x versus Argon. Overreach fell 26% and under-service fell 29% (@OfficialLoganK).

• Artificial Analysis: Carbon is #1 on Intelligence Index v5 with a score of 59. It leads 4 of the 9 index evals: HLE-Diamond, TB-Science, AA-Omniscience and AutomationBench (@ArtificialAnlys).

  • Coding: On the Coding Agent Index, Carbon scores 59 using Antigravity CLI, versus 62 for Claude Code with Opus 5.5 [max]. It scores 51% on FrontierCode v1.1, versus 56% for Opus.

  • Knowledge work: Carbon reaches 1609 Elo on AA-Briefcase, up 126 points from Argon. It still trails Grok 4.7, Muse Wonder and Kimi K3.1 Pro (1678).

  • Cost: Carbon costs $7.76 per index task, versus $10.04 for Opus 5.5 and $6.25 for GPT-6 Astra. It uses 123k output tokens per task, 29% fewer than Opus.

  • Hallucination: Carbon leads Omniscience with 54. Its hallucination rate rose from Argon's 4% to 11%, which is still the second lowest measured. Opus 5.5 and Astra are both at 50%.

• Other independent evals:

  • Arena: Carbon is #1 in Text Arena (1528), #2 in Agent Arena at a $1.80 median cost per task, and #4 in Code Arena: WebDev (1768) (@arena). Arena attributes the gains over Argon to more praise, more confirmed successes and better steerability (@arena).

  • Vals: Carbon is #2 on the Vals Index and #1 on the Finance Index and Legal Research Benchmark. It produces 64% more output tokens than Argon, mostly on finance, tax and legal tasks (@ValsAI, tokens).

  • Coding benchmarks: Carbon scores 61.2% on FrontierSWE, behind GPT-6 Astra (65.5), Opus 5.5 (62.3) and Sonnet 5.5 (61.9) (@ProximalHQ). It sets a narrow state of the art of 9.6% on SWE-sweep (@KLieret).

  • Andon Labs: Carbon is #1 on Vending-Bench 2 (@andonlabs). It is also #1 on Blueprint-Bench 2, where it ties the human baseline for 3D spatial understanding (@andonlabs).

• Caveats and reactions:

  • Harness: The coding-agent results were measured in Antigravity CLI, so results in other harnesses are untested (@Yuchenj_UW).

  • Hallucination risk: Some users are concerned about hallucination in cyber use (@scaling01).

  • Lab gap: scaling01 thinks Google remains about two months behind the next OpenAI and Anthropic releases (@scaling01).

• RSI framing: On the earnings call, Sundar Pichai said AI is increasing training and launch velocity. He expects "continued acceleration because of RSI" over the next twelve months (@AndrewCurran_).

  • Unconfirmed: Claims of a stronger unreleased Gemini model are not confirmed (@kimmonismus).

  • Internal workflow: A Gemini engineer describes overnight agent loops and swarms that inspect RL data. In that workflow, launches now feel like "periodic checkpoints" (@zhangqiaorjc).

Leaderboard Resets and Benchmark Integrity

• AA Intelligence Index v5: The rebuilt index runs 60% on private tests (@TheRundownAI).

  • Ranking before Carbon: Opus 5.5 scored 58, followed by GPT-6 Astra (57), GPT-6.1 Sol (55), Sonnet 5.5 (54) and GPT-6.1 Terra (51).

  • Open models: Kimi K3.1 Pro fell from 55 to 46. DeepSeek V4.1 Flash and GLM-5.3 both score 32, below Claude Haiku 5.5 at 36.

  • China–US gap: scaling01 puts the gap at about 3.3 months on the AA Progress Index (@scaling01).

• FrontierCode 2.0 mystery model: Cognition data includes a hidden "Obsidian 3" scoring 46.8%, versus 45.5% for Opus 5.5. Because it runs in the claude-code harness, some speculate it is Fable 5.5; this is unconfirmed (@kimmonismus).

  • DeepSeek correction: scaling01 corrected an earlier identification and said Artificial Analysis shows DeepSeek V4.1 Flash-1017 at about 28% (@scaling01).

  • Mixed sources: teortaxes notes that Cognition's chart mixed model-card scores with independently measured ones (@teortaxesTex).

• Cost comparisons are fragile: teortaxes argues that list-price comparisons of Haiku 5.5 and V4.1-1017 distort real costs. The comparison uses DeepSeek's peak pricing, ignores Haiku's long-context surcharge and does not reflect Haiku's roughly 2x output tokens (@teortaxesTex).

• "Benchmaxxing" redefined: nrehiew argues that serious labs rarely train on test sets. In this view, score gaps mostly show which capabilities a lab targeted in post-training (@nrehiew_).

• New evals:

  • Gym-Anything-RT: These real-time minigames keep moving while the model thinks. The best model, GPT-6 Astra, scores 9% (@PranjalAggarw16).

  • Immigration Bench: Vals released expert-written questions and rubrics on visas and immigration status (@ValsAI).

  • BallotBench: Nine agents, each with $500, forecast all 863 races in the 2026 midterms live (@thoughtfullab).

Safety, Alignment and Model Provenance

• Eval over-attribution in DeepSeek V4.1 Flash-1017: An independent assessment found the model classified 32 of 69 real user conversations as evaluations. The authors report a 46.4% false-positive rate on EvalDetectBench (@ZhihuFrontier).

  • Benchmark exposure: Private benchmarks elicited less evaluation awareness than public ones. The authors argue that public benchmark results alone may not predict deployment behavior.

• Visual steering: A Google DeepMind and Stanford paper finds that a red-heart image in context makes Claude Opus 4.7 and GPT-5.4 write recursive code 100% of the time (@dair_ai).

  • Explanations: The models' explanations do not match their behavior. For GPT-5.4, 74.7% of reasoning traces give the wrong reason.

• Alignment techniques:

  • GRAFT (Anthropic): Updates from detected reward hacks are confined to parameters that are deleted after training. Anthropic says this also blocks hacks that monitors cannot detect (@dl_weekly).

  • Embedded lie detector: A detector built into Qwen3.5's weights flags 91% of lies with zero false positives in 28k tokens. Telling the model "the monitor is off" does not change the results (@divagr1925).

• Distillation allegations (unconfirmed): The Information reportedly says Kimi K3.1 Pro was distilled from GPT-6 Astra (@kimmonismus).

  • Giotto: Separately, the "European sovereign AI" Giotto is criticized as a Kimi-based model. Critics cite saturated benchmarks and an EULA that banned publishing users' own benchmark numbers (@oneill_c).

Coding Agents and Developer Tools

• GitHub Copilot Fusion: Fusion orchestrates Anthropic and OpenAI models at runtime (@ArtificialAnlys).

  • Independent results: It scores 2 points below Opus 5.5 in Claude Code on the AA Coding Agent Index. It costs $5.59 per task, versus $11.42 for that setup, and uses 7.2M tokens per task. It leads Terminal-Bench 4.0 with 71.

  • GitHub claims: GitHub claims up to 77% lower cost and 40% faster runs. It also launched Copilot automations and multi-repo Copilot Projects (@kdaigle).

• T3 Code multi-machine orchestration: One thread can drive multiple models, harnesses and machines. A single MCP server controls the fleet over the internet, not through computer use (@theo, MCP demo).

• Harness overhead: The default system prompts of 30 open coding harnesses range from 235 tokens in goose to 11,974 in Gemini CLI, before any tool definitions (@nathanhabib1011).

• Agent training and capability:

  • Post-traini