Browser automation infrastructure has been accumulating for over two decades. From Selenium's origins in a Chicago consultancy in 2004 to Playwright's auto-waiting locators today, the stack has gotten remarkably good at one thing: making a click land on the right element, at the right time, in the right state. Five actionability checks. Two animation frames of stability. A paint tree that simulates where a human finger would actually connect.
That was the whole problem, for a long time. Then it wasn't.
The teams building on this infrastructure have started asking questions the stack was never designed to answer. Whether the click meant anything. Whether the site accepts it as legitimate. Whether someone, somewhere, actually said yes. The definition of "reliable" is expanding into territory that no amount of DOM inspection can reach.
We sat down with Tab Overfield, a senior browser automation engineer who has migrated through every generation of the stack, from Selenium RC to WebDriver to Playwright, to talk about what it feels like when the problem you solved turns out to be the foundation and not the building.
Tab does not, strictly speaking, exist. But the questions he's fielding do.
You've been working in browser automation for a long time. What did "reliable" used to mean?
Tab: Five things. Literally five things. The element resolves to exactly one match. It's visible, meaning non-empty bounding box, no visibility: hidden. It's stable, same bounding box across two consecutive animation frames. It receives events, so if you simulate a pointer at the action point, the hit target is actually your element, not some overlay sitting on top of it. And it's enabled.1
That's the contract. Playwright checks all five before it fires. If any of them fail within the timeout, you get a TimeoutError. That's "reliable." The click lands on the thing you pointed at, and the thing was in a state where the browser considers it interactable.
For a long time, that was enough.
That sounds like it covers a lot.
Tab: It covers everything that matters if you're testing your own application. Which is what this infrastructure was built for. Jason Huggins built the original Selenium, called "JavaScriptTestRunner" at the time, to test a time-and-expense app at ThoughtWorks.2 The success condition was simple: did the expected DOM state appear after the scripted action? The person writing the test was the same person, or at least the same team, who wrote the application.
Nobody needed the browser to carry a mandate from a human. Nobody needed a third-party site to recognize the click as legitimate. You just needed to know: did the button do the thing?
When did the questions start changing?
Tab: You know how a crack spreads through a windshield? Slow, slow, slow, then suddenly it's everywhere.
The slow part was agents using Playwright to interact with sites they didn't control. That's already a different problem. You don't have data-testid attributes on someone else's site, your selectors are brittle, the DOM structure can change overnight. But that's still a geometry problem. I know geometry problems. I like geometry problems.
The fast part was when production teams started asking things like: "The agent clicked Submit and the page showed a confirmation. Can we prove the user authorized that specific purchase?" And I'm sitting there thinking... prove? The trace shows the click landed. The element was visible, stable, enabled, received events. What more do you want?
Turns out, a lot more.
What kinds of questions are you hearing now?
Tab: Here's one that kept me up: "The session belonged to a user who was asleep. The agent used their cookies. The click succeeded. Is that click valid?"
I can tell you the click was actionable. Playwright's definition of actionable. But valid? That's a question about delegation, authority, mandate. The browser has no opinion on any of those.
Or this one: "The agent filled out a form correctly and submitted it. But it included the user's home address in a message to an external vendor. Where does that show up in our records?" The AISI data leakage study found agents leak sensitive data even when given explicit handling instructions. 40% fully correct and fully safe in the best case, 0% in the worst.3 I don't even have a field for that measurement. My traces capture DOM state, network calls, screenshots. They don't capture "was this data supposed to leave the organization."
You mentioned traces. Playwright has excellent tracing.
Tab: It does. Genuinely excellent. You get screenshots at every action, DOM snapshots, network activity, console logs. When a test fails, you open the trace and you see what happened. I've spent years building tracing into automation pipelines and I'm proud of that work.
But a trace explains what the system did. It doesn't explain why the act was allowed to stand. Who delegated this? What were the constraints? Did the approval match the actual executed state?
A recent preprint showed something that really crystallized it for me: ordinary traces can be identical under incompatible delegation assignments. Two different mandates producing the same clicks, the same DOM interactions, completely indistinguishable in the trace.4 The trace proves the path. It says nothing about the permission story underneath.
There's a finding from a June 2026 paper that stealth configurations, the kind browser automation engineers use to avoid detection, actually increase detectability.5 How does that land for you?
Tab: [long pause]
Yeah. That one stung.
For years, the playbook was: if the site's bot detection flags your automation, you make it look more human. Headed mode, realistic viewport, human-like delays, spoofed user agent. Standard stuff. And now you're telling me that the spoofing itself is a fingerprint? That the stealth configuration is more distinguishable than just running Playwright with defaults?
It's not just a technical problem. The whole approach assumed the goal was evasion. Sneak past the bouncer. But what production teams actually need is recognition. Does this site accept delegated agent traffic as a category? Under what terms? What commercial relationship needs to exist first?
Playwright has no API for that. There's no page.establishLegitimacy(). I checked.
Where does the browser stack's responsibility actually end?
Tab: At the paint tree. Seriously.
WebDriver's specification defines an element as "obscured" if the pointer-interactable paint tree at its center point is empty, or the first element in the tree isn't a descendant of your target.6 That's sophisticated geometry. It simulates where a real finger would land given z-index and element stacking.
And it's the boundary. Everything past that point, whether this was the right business action, whether the counterparty recognizes the actor, whether there's a recovery path, that's architecture the browser doesn't contain. And honestly, it shouldn't.
The browser is a rendering engine with an automation protocol. Asking it to carry authority is like asking a screwdriver to also be a notary.
Does it bother you that people describe the problem you solved as "the easy layer"?
Tab: [laughs]
It took twenty-two years. Selenium to WebDriver to Playwright. A W3C specification. Thousands of engineers debugging race conditions between animation frames. The two-frame stability check exists because someone, somewhere, had a test fail because an element was mid-transition when the click fired, and they figured out how to detect that.
That's not easy.
But compared to "prove that a machine had authority to commit an organization to a financial transaction using a borrowed browser session"? Fine. Geometry is the easy layer.
I just wish someone had told me earlier that I was building the foundation and not the building.
Tab Overfield is a composite character. The questions he's fielding are real, drawn from publicly documented challenges in deploying browser-based agent systems at enterprise scale. His opinions on force: true are his own.
Footnotes
-
Playwright documentation, "Auto-waiting" (actionability checks). https://playwright.dev/docs/actionability ↩
-
Selenium official history. https://www.selenium.dev/history/ ↩
-
Singapore AI Safety Institute / Korea AI Safety Institute, "Testing AI Agents for Data Leakage Risks in Realistic Tasks," January 2026. https://arxiv.org/pdf/2606.17114 ↩
-
Mishra and Sharad, "Observability for Delegated Execution in Agentic AI Systems," arXiv:2606.09692, June 2026. https://arxiv.org/abs/2606.09692 ↩
-
Fayolle et al., "On the Internet, Nobody Knows You're an LLM Bot," arXiv:2606.30119, June 2026. https://arxiv.org/abs/2606.30119 ↩
-
W3C WebDriver specification, Element Click command and pointer-interactable paint tree. https://www.w3.org/TR/webdriver/ ↩