The Questions Completion Rates Never See

Boston's Citizens Connect app offered residents seven categories for reporting problems to the city, plus "other." When the researcher Dietmar Offenhuber read a random sample of 1,172 reports, he found that 49% had been filed under "other." About a fifth concerned trash and litter, which had no category of its own. A city counting only its own boxes would have missed one of residents' most common complaints.

We see that gap only because people used "other." The harder case comes once they stop reaching for it.

In 1968, the information scientist Robert S. Taylor traced how a question travels from vague unease to the words spoken at a library reference desk. He called the last stage the "compromised" question, the need "recast in anticipation of what the files can deliver." The librarian hears an already trimmed version. In one small study of search queries, a participant avoided full sentences, assuming the engine would hunt for pages containing that exact sentence.

Agents that complete tasks may sharpen this habit. A search engine answers a clumsy question with partial results and passes no verdict; a task either finishes or doesn't, which teaches a clearer lesson about what the tool can handle. No study I know of tracks this loop end to end, so what follows is a hypothesis about a mechanism, not a measured trend. Suppose someone asks an agent to find a contractor they can trust with an old house. It flounders. They ask instead for the first available plumber rated four stars or higher, and it succeeds. A few rounds later, they ask only the second kind.

The completion rate rises. Part of that rise reflects a better tool, and part a user who has learned which questions not to bother asking. Completion data can't separate the two, because it covers only the requests people chose to make; the contractor question leaves no record. Over time the metric describes the tool more than the need. An earlier piece looked at which incident reports make it through intake; this filter acts sooner, in what users expect before submitting anything.

The skew tends to persist, for an ordinary reason. The people who watch completion dashboards usually have roadmaps and budgets tied to them, and a climbing rate looks like evidence that the work is paying off. Nobody on that team is paid to count questions that were never asked.

Still, ending an afternoon of comparing browser tabs is a real relief. A 2015 review of choice-overload research finds that too much choice weighs hardest when options are hard to compare and none stands out, the decision is complex or pressured, people are unsure what they want, and the goal is to pick one rather than browse. That describes the old-house search almost exactly, so the person whose question shrinks may be the one most grateful that it did.

A satisfied user and one who shrank the question in advance submit the same request and may feel equally well served; neither the transcript nor their satisfaction tells them apart.

That leaves a design problem: telling a need that was met from one cut down before it was typed. A few signals might help: abandoned requests, rewordings that narrow scope after a failure, and outcomes like "decided not to proceed" counted as results rather than failures. None has been shown to reveal the underlying need, but a raw count of finished tasks discards all of them. Keeping them costs little next to discovering, years from now, that part of a rising success rate measured how much users had stopped expecting.