The TRUTH about Sakana Fugu..

Fugu is a learned orchestration API, not a model — and the benchmarks backing it don't hold up under scrutiny.

What Fugu actually is

Sakana frames Fugu as a single API that "learns to orchestrate across a pool of frontier models" — handling model selection, delegation, verification, and synthesis automatically. Vercel added Fugu Ultra to its AI Gateway the same day it was released, which is the strongest early signal that platform teams see value in the abstraction. Audrey Tang reported that Fugu Ultra worked well as a planner paired with a fast driver loop, fitting the slow-planner / fast-executor architecture many agent stacks already use.

Where the model actually sits

Fugu is not pitched against Claude, Gemini, or Qwen as a direct replacement. It targets engineers who already run multi-step agent workflows and manually switch models, or who build learned routing into their own harnesses. Sakana demonstrated the approach on auto-research, finance, blindfold chess, and computer-aided design as test-time coordination workloads.

Why the release got torn apart

Ellie (ex-Hugging Face) published a four-point critique: Fugu trails Claude Opus on SWE-bench Pro by roughly 10 points; Sakana compares against anonymized Model A, B, C instead of named baselines; no token or cost reporting is provided despite best-of-N orchestration burning inference budget; and the fair comparison should be against other test-time scaling methods like parallel voting, multi-agent debate, or self-consistency. Blanche Minerva also publicly challenged Sakana's trustworthiness, citing prior incidents and "impossible performance claims in earlier work."

What this means for builders

Aaron Levie (Box CEO) framed the thesis: "routing and orchestration is likely the high-value layer, not the base model." Fugu's release proves demand exists. The backlash proves that shipping an orchestration system in 2026 without disclosing cost, routing overhead, token burn, and a proper test-time scaling baseline is no longer acceptable. The opaque orchestration benchmark era is closing.