CURRENT

GAIA evaluates general-purpose reasoning across web browsing, tool use, and multimodal tasks. What it won't tell you: how much credit belongs to the model. Swapping scaffolds shifts scores by roughly 30 points, far more than upgrading between frontier models. One run costs $2,829 before caching. When someone cites a GAIA number, the first question is which framework produced it.

Claude Opus 4.5 hits 80.9% on SWE-bench Verified and 45.9% on Pro. Three things compound: possible contamination (models reproducing verbatim patches), harder tasks in Pro requiring 10+ line changes, and Pro's controlled harness stripping away scaffold advantages. Verified measures something. Pro measures something closer to your actual codebase.

Sierra's τ-bench runs agents through customer service conversations, then runs them again. And again. GPT-4o passes a single attempt over 60% of the time. Require it to pass the same task across eight runs, and that number collapses below 25%. Most benchmarks report the generous number. τ-bench reports both, and the gap between them is the reliability problem production teams actually live with.

Online Mind2Web tests agents on live websites that change underfoot. Browser-Use with Claude Sonnet 4 spent $1,577 to reach 40% accuracy. SeeAct with GPT-5 Medium hit 42% for $171. Two percentage points apart, ninefold cost difference. The benchmark measures web task completion. It accidentally also measures how wildly scaffold-model pairing affects economics.

Princeton's Holistic Agent Leaderboard burned $40,000 on 21,730 rollouts across 9 models and 9 benchmarks, standardizing scaffolds and tracking costs that individual leaderboards quietly omit. Independent sweeps revealed a 33x cost spread on identical tasks from scaffold choice alone. HAL has since paused accuracy updates entirely, pivoting to reliability measurement. Even the meta-benchmark decided the headline numbers weren't the point.

If your hardest problem is knowing exactly what happened at each step of a branching, retrying workflow, LangGraph's graph state machine saves typed snapshots at every super-step. Time-travel debugging and human-in-the-loop interrupts are built into the core abstraction, not layered on top.

If your hardest problem is tight coupling with Claude models, the Claude Agent SDK gives you four extension points wired directly into the model loop: MCP for tools, skills for reusable procedures, hooks for lifecycle control, subagents for parallel isolated work. Same architecture that runs Claude Code.

If your hardest problem is getting specialized agents to coordinate without hand-drawing graph topology, CrewAI lets you define agents by role, goal, and backstory. Delegation happens across sequential, hierarchical, or consensual patterns. You can ship a working multi-agent system before sketching a single node.

If your hardest problem is enterprise buy-in within Azure, Microsoft Agent Framework 1.0 (GA April 2026) merges Semantic Kernel and AutoGen into a single .NET and Python SDK. Both predecessors are now in maintenance mode. Swapping between six model providers takes one line of code.

If your hardest problem is workflows that wait on humans for days or die mid-run, durability outranks every other feature. LangGraph checkpoints natively at every graph boundary. Microsoft Agent Framework 1.0 supports pause-and-resume across all orchestration patterns. Without durability, a crashed agent restarts from zero.

If your hardest problem is triaging incoming requests to the right specialist agent with safety checks in place, OpenAI's Agents SDK treats handoffs and guardrails as native primitives with automatic tracing. Code-first, no graph to predefine. The tradeoff: less portable, and less suited to stateful long-running work.