Vision — Issue 33

The Core Problem

The Quiet Part

By Nora Kaplan— June 11, 2026

Feature image for article: The Quiet Part

The Core Problem

The Quiet Part

By Nora Kaplan— June 11, 2026

Lessons and Designs

What Aviation Built After It Noticed the Problem

Aviation solved the automation-complacency problem decades ago with simulator checks, mandatory manual flying, and crew resource management. Those countermeasures rest on a foundation enterprise AI lacks: a stable, agreed-upon definition of what correct human performance looks like. A pilot's hand-flown approach can be graded against known parameters. An analyst's judgment on a market entry recommendation cannot. That structural difference determines whether countermeasures are even possible.

Lessons and Designs

Monitoring the Monitors

Organizations are building increasingly sophisticated systems to monitor whether their AI agents perform well. Almost none are checking whether the humans evaluating those agents still can. Evidence from the few domains where measurement exists — developer productivity, medical diagnostics — reveals a durable gap between perceived and actual human capability. The instrument to detect oversight degradation needs to be built now, while the people who could design it still have the expertise to know what it should measure.

Lessons and Designs

What Aviation Built After It Noticed the Problem

Aviation solved the automation-complacency problem decades ago with simulator checks, mandatory manual flying, and crew resource management. Those countermeasures rest on a foundation enterprise AI lacks: a stable, agreed-upon definition of what correct human performance looks like. A pilot's hand-flown approach can be graded against known parameters. An analyst's judgment on a market entry recommendation cannot. That structural difference determines whether countermeasures are even possible.

Lessons and Designs

Monitoring the Monitors

Organizations are building increasingly sophisticated systems to monitor whether their AI agents perform well. Almost none are checking whether the humans evaluating those agents still can. Evidence from the few domains where measurement exists — developer productivity, medical diagnostics — reveals a durable gap between perceived and actual human capability. The instrument to detect oversight degradation needs to be built now, while the people who could design it still have the expertise to know what it should measure.

Lessons and Designs

What Aviation Built After It Noticed the Problem

Aviation solved the automation-complacency problem decades ago with simulator checks, mandatory manual flying, and crew resource management. Those countermeasures rest on a foundation enterprise AI lacks: a stable, agreed-upon definition of what correct human performance looks like. A pilot's hand-flown approach can be graded against known parameters. An analyst's judgment on a market entry recommendation cannot. That structural difference determines whether countermeasures are even possible.

Lessons and Designs

Monitoring the Monitors

Organizations are building increasingly sophisticated systems to monitor whether their AI agents perform well. Almost none are checking whether the humans evaluating those agents still can. Evidence from the few domains where measurement exists — developer productivity, medical diagnostics — reveals a durable gap between perceived and actual human capability. The instrument to detect oversight degradation needs to be built now, while the people who could design it still have the expertise to know what it should measure.