Dag Halvorsen does not exist. He is a composite, assembled from peer-reviewed interview data, survey findings, and the observable patterns of how enterprise teams actually operate agent systems in production. The pressures he describes are real. The person is not. If you find yourself nodding along anyway, that's the point.
A recent ICML study found that 47% of deployed agents execute fewer than five steps before a human must intervene.1 That number sounds like a limitation. To the people who set those limits, it sounds like a policy.
Dag Halvorsen is a Senior Staff Engineer at a large U.S. insurance carrier, where he oversees what he calls "agent reliability," a title no one gave him and no one has taken away. Before agents, he spent a decade in site reliability engineering. He now manages a portfolio of internal agents handling claims triage, prior-authorization support, and document extraction. We spoke over video. His office had a whiteboard behind him covered in what appeared to be a flowchart that had been partially erased and redrawn several times. Some of the arrows pointed in both directions.
You came from SRE. What carried over?
Dag: The error budgets, honestly. In SRE you have this concept where you're allowed a certain amount of downtime, and you spend it deliberately. With agents, I do something similar, except the budget isn't downtime. It's wrongness. How much wrongness can this workflow tolerate before someone gets hurt or we get sued? That number determines everything else. Step counts, review rates, which tasks we even attempt.
So when you set a step limit, say five steps, how do you arrive at that number?
Dag: I wish I could tell you there's a formula. There isn't. You look at the task. Claims triage. The agent reads a submission, pulls structured fields, checks them against policy rules, flags discrepancies. That's four steps. I set the limit at five because sometimes it needs to re-query a document. That's it. The number comes from watching the task, not from any optimization.
The honest answer is it's a heuristic that gets encoded as configuration, and then everyone treats it like it was derived from first principles.1
The same study found 75% of production teams evaluate without formal benchmarks. What does evaluation look like for you?
Dag: [leans back, folds arms] Golden sets. We have, I think right now, about 340 question-answer pairs for claims triage, built over seven months. Our domain experts labeled them. Seven months. And they're not transferable to the prior-auth agent because different policy logic, different edge cases, different everything.
So we run the golden set when we change anything. We also sample production output. Our LLM judge scores confidence on every response, and anything below threshold goes to a human reviewer. Then we randomly sample maybe 8% of the high-confidence ones too.1
Why 8%?
Dag: Because that's what our reviewers can handle in a day without burning out. It's a staffing number, not a statistical one. I've never admitted that in a meeting.
That's remarkably candid.
Dag: It's remarkably common. I just don't think people say it out loud.
You mentioned domain experts spent seven months building your evaluation set. What happens when someone asks you to extend the agent to a new task class?
Dag: That's the conversation I have roughly every six weeks. Someone sees the claims triage numbers, we're about 12x faster than manual processing, and says great, let's have it do subrogation analysis too. And I have to explain that "ready" isn't a switch I flip. We'd need new golden sets, which means expert time we don't have. We'd need to understand the failure modes specific to that task. We'd need to run it internally first, read-only, before it touches anything real.1
The pattern is always: internal before external, read-only before write-access, sandbox before production. Each stage needs enough evidence to justify the next one.1 And in insurance, the feedback on whether you got something wrong can take months. A bad triage decision doesn't surface until a claim is disputed or an audit happens. So I can't iterate fast. I can't build confidence fast. I'm operating on delayed, noisy signals while everyone around me is looking at the speed metrics and asking why we're not doing more.
You're describing political pressure.
Dag: I'm describing a productivity narrative that creates its own gravity. Eighty percent of agent deployments are justified by productivity gains.1 That's the story the organization tells itself. And it's true, the agent is faster. But faster at what confidence level? Over what task distribution? With what review capacity behind it?
Nobody asks those questions when the demo goes well. They ask them after the first bad outcome, and by then the system's been running for three months at a step count I argued against.
What about model upgrades? Your carrier presumably isn't training its own models.
Dag: [exhales] So this is what actually keeps me up at night. We built seven months of evaluation confidence on a specific model's behavior. Then the provider ships an update. The benchmarks say it's better. And maybe it is, on their benchmarks. On our golden set, with our prompts, it might be worse in ways that take weeks to surface.
We're currently running two model versions simultaneously because the newer one broke edge-case handling in document extraction.1 Nobody planned for that. That's operational debt from not controlling the thing your entire system depends on. It's like renting the engine in your own car.
Your system prompt, how long is it now?
Dag: [glances at something off-screen, winces slightly] The claims triage prompt is around 9,000 tokens. It started at maybe 400. Every incident, every edge case, every compliance requirement gets folded in. It's less a prompt now and more of a policy manual written in natural language that I can't unit test.1
I added a clause last month about handling Puerto Rico-specific regulatory language. It interacts with the general U.S. regulatory clause in ways I'm not fully confident about. But I can't test every interaction. I test the golden set and hope the coverage is sufficient. Hope is doing a lot of load-bearing work in that sentence.
How do you know when something is actually working?
Dag: Time-to-completion versus the human baseline. That's the metric everyone looks at.1 And it's real, it matters. But it's the easiest thing to measure, not the most important thing to measure. Correctness, safety, data handling: those are harder and slower to surface. Especially in insurance, where the consequences of being wrong are financial and sometimes show up two quarters later.
The 2025 AI Agent Index looked at 30 prominent deployed agents and found that 25 disclosed no internal safety results publicly.2 I don't find that surprising. It's not that people are hiding things. It's that the internal confidence is built on processes, golden sets, sampling, expert review, that don't translate into a number you'd put on a system card. You can't publish a vibe.
Last question. Do you ever feel like you're holding back something that should be let loose?
Dag: [long pause] No. I feel like I'm holding back something that will be let loose, eventually, when the evidence justifies it. My job is to make sure "eventually" isn't "next Tuesday because the VP saw a demo."
The constraints aren't the problem. The constraints are the product. Everything else is aspiration.