A frontier lab shipped its enterprise agent product last week with a disclaimer attached: not yet available as a self-serve product. Deployments run through the lab's own forward-deployed engineers and a short list of systems integrators.
That is rationing, and the rationed good is not model access. Models are abundant. What's scarce is whoever scopes the job, writes the policies, and decides where a human takes the call back.
The ICML production study suggests why. Most agents surveyed run ten steps or fewer before a person intervenes. Roughly three-quarters still depend primarily on human evaluation. Reliability, practitioners report, gets solved at the systems level, which is another way of saying it gets solved by people.
That context does something to the 75% unassisted-resolution figure. Company-reported, no denominators, no outside grader. It describes what a lab achieved on its own support line, with its own engineers, on a scope it picked.
Watch where a lab spends its engineers. That constraint doesn't move on a release schedule.
What ships with the agent: one defined job (billing, claims, IT tickets), only the access that job needs, customer-set approval and takeover rules, plus simulation and eval tooling.
Who installs it: vendor forward-deployed engineers and select global systems integrators. Eligible enterprise customers only.
The two numbers offered: 75% of inbound issues resolved without human assistance; handoffs down 15 points in 10 days. Both from the vendor's own English-language phone line.
Not disclosed: call volume, the definition of "resolved," grader methodology, independent verification, steady-state window.
From the ICML survey (20 case studies, 306 practitioners, 26 domains): 68% of production agents cap at ten steps before intervention · 70% prompt off-the-shelf models rather than tune weights · 74% lean on human evaluation.
A counterweight: a controlled field study of 647 support agents found faster chats alongside lower customer ratings. Fewer handoffs and better outcomes are separate measurements.