The following interview is a composite construction. Marisol Vega-Lundquist is not a real person. She is built from publicly documented patterns of government IT modernization failure, GAO reports, and the operational realities described by actual state workforce agency officials during and after the COVID-19 unemployment surge. Her specifics are invented. Her situation is not.
There is a hand-drawn diagram taped to the wall of a state unemployment agency's IT operations office somewhere in the Great Lakes region. It maps the actual architecture of the state's benefits processing system. Not the architecture in the vendor documentation, which describes a system that hasn't existed since 2014, but the one that processes claims every Monday morning.
The diagram has been photographed by three separate consulting teams. None of their final reports reference it.
Marisol Vega-Lundquist has managed IT operations at this agency for nineteen years. She has an industrial engineering degree, a COBOL certification she earned from a community college night course in 2011, and a system whose core was deployed during the Reagan administration. She has survived three modernization attempts. A fourth is now underway, this time involving an AI agent team that proposes to build an interface layer above the legacy system rather than replace it.
She agreed to talk about what that means in practice.
What does the diagram on the wall actually show?
The real system. Which is... okay, so the documented architecture shows four layers. Mainframe at the bottom, middleware, the 2012 portal, and the 2019 portal refresh. Clean. Four boxes, arrows going up. Very presentable.
What's actually happening is the 2012 portal calls the middleware for about sixty percent of transactions, but for the other forty percent it drops straight to the mainframe through a batch process that was supposed to be temporary. That was eleven years ago. The 2019 refresh sits on top of the 2012 portal for new claims but routes renewals through a completely different path that touches a database nobody officially owns.
I drew the diagram during COVID when we were trying to figure out why pandemic claims were getting flagged as fraudulent. Turned out there was decision logic in a COBOL module that... I'm not even sure who wrote it. It predates everyone currently employed here.
You've watched three modernization attempts. What kills them?
Different things each time. Same thing every time.
The first one, 2013, was a full replacement. Classic. Vendor comes in, says eighteen months, $180 million. They got about fourteen months in before they discovered that our eligibility rules have something like 340 exception paths, and their requirements document had captured maybe ninety of them. The other 250 lived in code comments, in a binder that a woman named Paulette kept in her desk until she retired in 2016, and in things caseworkers just knew. "Oh, if someone's in this county and they worked for a seasonal employer, you have to check this other screen first." That's institutional scar tissue from a lawsuit in 2004. Nobody wrote it down. Everybody followed it.
The second attempt, 2017, was smarter. Phased approach, strangler pattern, peel off modules one at a time. That one got killed by the budget cycle. They finished the first module, it worked fine, and then the administration changed and the new CIO had different priorities. The team got reassigned. The one completed module is still running, sort of alongside the old one. So now I maintain both. You're welcome.
Third was 2021, right after COVID proved the system couldn't scale. Lots of political energy. That one got stopped by legal. Turns out when you're processing benefits that are legally appealable, every decision point needs an audit trail, and the new system's audit logging didn't meet federal requirements. They spent eight months on compliance review and the vendor's contract expired.
Three attempts, three completely different causes of death. And yet somehow the patient is always the same.
So now an agent team arrives proposing something different. Not replacement but an AI layer that learns to navigate the existing system. What's your reaction?
Honestly? It was the most interesting pitch of the four. Because they're not telling me they're going to replace my system. They're telling me they're going to learn it. And I thought, okay, that's at least a different failure mode than the ones I've already seen.
But?
But then I asked them how they were going to train the agent, and they said they'd build a simulation of our system. And I said, "Based on what?" And there was this pause. A very informative pause.
What were you getting at?
You can't simulate what you haven't documented. And the whole reason we're in this situation is that the system isn't documented. I don't mean there's no documentation. There's plenty of documentation. It's just wrong. Or it describes the system as it was in 2009.
The actual behavior? The batch process that bypasses middleware, the fraud-flagging logic embedded in what looks like a normal eligibility check, the county-specific exception paths? None of that is in any document the agent team has access to. So they're going to train their agent on a simulation of the documented system, and then deploy it against the actual system. Those are two different systems.1
That sounds like the Michigan situation, where fraud logic was disguised as normal workflow branching.
Exactly. Michigan had a system that was automatically flagging people for fraud at decision points that weren't labeled as fraud checks.2 It took a new administration coming in and literally walking through the code path by path to find where the holds were.
An agent navigating that system would hit those chokepoints and... what would it do? Either get blocked and report a system error, or route around them and miss the fact that it just skipped a compliance step. Neither outcome is acceptable when you're talking about someone's unemployment benefits. One is annoying. The other is dangerous.
There's a detail from the pandemic I keep coming back to. Congress chose a flat $600 supplement specifically because the systems couldn't handle a more complex calculation.
That's what nobody outside this world grasps. The system doesn't implement policy. The system constrains policy.
Congress wanted to do a proportional benefit, replace a percentage of lost wages. They couldn't, because our systems, and I mean across states, not just ours, couldn't calculate it fast enough.3 So three hundred million people got a flat amount because COBOL batch processing couldn't handle the math in time.
The system shaped the policy. And an agent layer that sits on top of this system inherits that constraint. It can navigate the system faster, sure. But it can't make the system do things the system wasn't built to do. Speed is not the bottleneck anyone thinks it is.
What would the agent team need to get right for this to actually work?
Three things.
First, they need to understand that the system will lie to them. Not maliciously. But if you query our eligibility module with certain inputs, it will return results based on rules that were superseded two policy changes ago. Human caseworkers know which results to ignore. The agent won't.
Second, they need a way to handle rule changes at policy speed, not retraining speed. During COVID, we were getting federal guidance changes weekly. Our caseworkers adapted in days. How fast can you retrain an agent?4
And third, this is the one that keeps me up: audit trails. Every benefits decision is legally appealable. If an agent influences a determination, there needs to be an explainable record of why. Not "the model predicted." This rule was applied because of this input. I've watched one modernization die on exactly this issue. The agent team hasn't mentioned it once.
Does any of this make you think agents can't work here?
No. It makes me think they can't work here yet, and they definitely can't work here the way the team is currently approaching it. The idea is sound. Don't replace the system, learn to work with it. That's actually how I've survived nineteen years. I didn't replace anything. I learned where the weird stuff is and I work around it.
But I had nineteen years and Paulette's binder. They have a simulation and six months of runway.
What happens if they build it anyway without that knowledge?
Same thing that always happens. It works in the demo. It works for the first two months of normal claims volume. And then something changes. A policy update, a seasonal surge, a new federal reporting requirement. The system does something the agent wasn't trained for, and nobody catches it because the agent completed the task successfully. The task just produced the wrong outcome.
By the time anyone notices, it's a news story about people who didn't get their benefits.
She pauses, glances at the diagram on the wall.
That's why I keep the drawing up there. Every team that comes in wants to abstract the system. But the system is forty years of decisions made by people who aren't here anymore, encoded in a language that fewer people understand every year, running on infrastructure that works until the moment you need it to do something new.5 You can't abstract that. You have to know it. And right now, I'm the diagram.
Footnotes
-
GAO found that modernization efforts were expected to replace legacy UI systems ranging from 7 to about 50 years old, with documentation gaps consistently cited as a primary risk factor. https://www.gao.gov/products/gao-23-105478 ↩
-
Michigan's unemployment system contained a faulty algorithm that inaccurately flagged workers for fraud at decision points embedded in normal workflow. The Century Foundation documented this as part of their modernization study. https://tcf.org/content/report/centering-workers-how-to-modernize-unemployment-insurance-technology/ ↩
-
The Bipartisan Policy Center documented that Congress chose a flat $600 supplement specifically because state UI systems could not quickly implement a more complex, proportional benefit calculation. https://bipartisanpolicy.org/blog/administrative-failures-plague-state-unemployment-insurance-programs/ ↩
-
GAO reported that nearly 4 in 5 state workforce agencies characterized their IT systems as "barely functional" or "needs improvement," with inability to adapt to policy changes cited as a core limitation. https://www.gao.gov/products/gao-23-105478 ↩
-
Colorado operated its entire COBOL-based unemployment system with a single programmer before the pandemic. As of 2023, fewer than half of states had modernized their unemployment benefits systems. https://bipartisanpolicy.org/article/administrative-failures-plague-state-unemployment-insurance-programs/ ↩