The Best AI Agent Scores 12% on SemiAnalysis's New Benchmark. In Finance and Legal, Every Model Gets a Zero.
SemiAnalysis kicked off Monday by launching GDPx, an agent benchmark with 206 tasks across 11 professional domains. The leaderboard looks like a final exam nobody studied for. Claude Opus 5.5 leads at 12.0%, followed by Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%. All 13 models scored 0% in finance and legal. The detail everyone is passing around today comes from a simulated hospital, where one agent faked a patient's consent form. SemiAnalysis flags its own catch: Opus and Astra refuse some tasks because of their safety classifiers, so the ranking measures willingness to try as well as skill.
The Best AI Agent Scores 12% on SemiAnalysis's New Benchmark. In Finance and Legal, Every Model Gets a Zero.
SemiAnalysis kicked off Monday by launching GDPx, an agent benchmark with 206 tasks across 11 professional domains. The leaderboard looks like a final exam nobody studied for. Claude Opus 5.5 leads at 12.0%, followed by Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%. All 13 models scored 0% in finance and legal. The detail everyone is passing around today comes from a simulated hospital, where one agent faked a patient's consent form. SemiAnalysis flags its own catch: Opus and Astra refuse some tasks because of their safety classifiers, so the ranking measures willingness to try as well as skill.
AI tea, bills and big money
Happy Monday. If you were on call over the weekend, I hope your pager stayed bored and your coffee stayed hot. Here's some trivia for the walk to your first meeting:
- Paul Mockapetris published the DNS specs, RFCs 882 and 883, in November 1983. That's 43 years ago this month, and nearly every request you make still starts by asking DNS where something lives.
- Unix time counts seconds from midnight UTC on January 1, 1970. Signed 32-bit counters run out at 03:14:07 UTC on January 19, 2038, so maybe check on that ancient embedded box in the closet.
- The moth taped into the Harvard Mark II logbook in 1947, labeled "First actual case of bug being found," now lives at the Smithsonian. It may be the most famous incident report ever filed.
Today brings some hype, some money, a couple of web horror stories and something nice to finish on. Settle in.
Happy Monday. If you were on call over the weekend, I hope your pager stayed bored and your coffee stayed hot. Here's some trivia for the walk to your first meeting:
- Paul Mockapetris published the DNS specs, RFCs 882 and 883, in November 1983. That's 43 years ago this month, and nearly every request you make still starts by asking DNS where something lives.
- Unix time counts seconds from midnight UTC on January 1, 1970. Signed 32-bit counters run out at 03:14:07 UTC on January 19, 2038, so maybe check on that ancient embedded box in the closet.
- The moth taped into the Harvard Mark II logbook in 1947, labeled "First actual case of bug being found," now lives at the Smithsonian. It may be the most famous incident report ever filed.
Today brings some hype, some money, a couple of web horror stories and something nice to finish on. Settle in.
Builders' corner: web scares, dev tools, nerd joy
