Anthropic's Claude Sonnet 4.5 scores 61.4% on OSWorld, the benchmark for real-world computer tasks. That's the highest score we have for autonomous desktop interaction. It's also nowhere near production-ready.
The number itself is the signal. What it reveals is the gap between capability and deployment. We've spent the last year building infrastructure that keeps browser agents running reliably across thousands of websites, each with different authentication flows, bot detection systems, and DOM structures that shift without warning. That experience shapes how we read this benchmark. Model capability is one thing. Building production systems on top of something that fails 40% of the time in controlled conditions is another.
What Web Automation Taught Us About Failure
When we see that Claude struggles with scrolling and dragging, we recognize the pattern from web automation. These aren't bugs to fix. They're fundamental challenges of translating intent into precise interactions.
On the web, "click the login button" can mean navigating through modal overlays, handling cookie consent, and dealing with buttons that move as the page loads. Desktop environments have their own version of this complexity. That 61.4% suggests it's just as hard.
A 40% failure rate doesn't mean you succeed 60% of the time and fail 40%. It means you need infrastructure that handles cascading failures, clustered error patterns, and real-time decisions about retry, escalation, or abort.
Production teaches you to distinguish between "the agent couldn't find the button" and "the interface changed" and "the session hit unexpected authentication." We've learned that failure modes on the web are adversarial. Sites actively resist automation. Desktop environments might be more cooperative, but that 61.4% suggests similar complexity lurking in what seems like straightforward interaction.
The Infrastructure That Failure Demands
Anthropic knows this. Their documentation explicitly states:
"Computer use remains slow and often error-prone"
They recommend starting with "low-risk tasks" in trusted environments. They note that scrolling, dragging, and zooming present challenges for Claude. Actions people perform effortlessly. These aren't edge cases. They're fundamental interactions that happen constantly in real workflows.
The infrastructure requirements aren't optional:
- CloudWatch metrics for visibility
- SAML authentication for access control
- Sandboxed execution environments for containment
- Programmatic usage tracking for audit trails
Multiple vendors now offer security solutions specifically for Claude integrations. That's a market that exists because the capability creates new risks that need containment.
When enterprises ask us about deploying agents, they focus on success rates. What they really need to understand is what happens during the failures. Production systems live or die there. Can your infrastructure handle the agent hallucinating coordinates, making unexpected tool selections, or taking actions you didn't anticipate? Can you catch those problems before they cascade? That's what matters.
Where Better Models Take Us
The benchmark has improved rapidly. Claude scored 22% in October 2024, then 42% by mid-2025, now 61%. That trajectory suggests we'll see 80%, then 90%, eventually 95%.
Better models don't eliminate monitoring requirements. They don't make error handling optional. They don't remove the need for audit trails or security controls. If anything, as capabilities improve, infrastructure requirements intensify. We've watched this evolution in web automation.
As agents got better at navigating sites, enterprises didn't need less infrastructure. They needed more sophisticated infrastructure. Better agents meant customers built more critical workflows around them, which meant failures had bigger impact, which meant monitoring and governance became more essential.
Computer use agents will follow the same path. A 22% success rate limits what you'd even attempt. A 61.4% success rate makes more workflows seem possible, which means more scenarios where things can go wrong in consequential ways.
Will the models improve? Yes. Will the infrastructure exist to make that improvement production-ready? That's the actual question.
What matters isn't the computer use feature itself. What matters is what the reliability gap reveals about market direction. Enterprises don't need better demos. They need infrastructure that makes unreliable capabilities dependable enough to build operations around. That's where value crystallizes. Not in the agent's ability to click buttons, but in the systems that make those clicks reliable when it matters.
-
OpenAI's Operator comparison: While Claude leads on desktop interaction at 61.4%, OpenAI's Operator achieved 87% on WebVoyager (web navigation tasks) compared to Claude's 56%, though Operator remains limited to browser-based tasks.
-
The efficiency problem: Even when agents succeed, research shows they take 1.4× more steps than required compared to human-generated minimal trajectories, with the best agent achieving only 17.4% on strict efficiency metrics.
-
Real production costs: Users report that asking Claude to open and categorize a single URL cost approximately $1.30, highlighting the token consumption challenges that come with screenshot-based computer interaction at scale.
-
Security infrastructure market: The emergence of multiple vendors offering security solutions specifically for Claude integrations signals that computer use capabilities create containment requirements enterprises can't ignore.