The Routing Decision Nobody Governs

Governance assumes a subject. Attribution needs a "who." Authorization needs an entity whose behavior was the thing you approved. Every compliance framework, every audit trail, every accountability chain eventually points at something and says: this reasoned, this decided, this bears responsibility. For AI agents, that something is "the agent," and until recently the approximation held well enough. Multi-model routing has quietly dissolved it.

In production agent architectures today, a planning step might run on one model, execution routes to another, a validation pass calls a third. A practitioner account documents the logic plainly: code generation queries go to technically proficient models, everyday questions route to cost-effective locally-hosted models (partly to keep sensitive data on premises), creative tasks requiring larger context windows go to premium models. Each of these models has different training data, different failure modes, different behavioral tendencies. The routing function examines state or cost or complexity and decides which reasoning character shows up at each turn.

The trace tells a partial story. The OpenTelemetry GenAI conventions define an agent with four identity attributes: description, ID, name, version. None of them include which model the agent uses. Model identity lives one layer down, at the individual LLM call span, captured only when a specific API request fires. To know which model reasoned through a particular step, you walk the span tree after the fact and reconstruct the answer from call-level metadata. The routing decision itself, the moment where one reasoning character was chosen over another, is not a named event type in any major framework's trace. You can see which node executed. You cannot see why that model was selected.

Attribution and authorization compound from here. IBM has flagged a temporal version of the problem: by the time someone asks why an agent decided something, the model that decided may have been updated or deprecated. Multi-model routing makes this worse within a single workflow run. The model that planned an action and the model that executed it are different reasoning characters wearing the same name. Because attribution is incomplete, authorization review is always retrospective and always uncertain. When an enterprise grants an agent permission to act, that approval implicitly assumes something about how the agent reasons. If the routing layer swaps in a different model mid-workflow, the behavioral profile the authorization was scoped to has shifted without any authorization event firing. Analysis of the EU AI Act has identified that when multiple providers contribute to a single agent's output, the question of which provider bears accountability is genuinely unresolved. The Act's framework was designed for a world where "the model" was singular.

The OpenTelemetry agentic conventions proposal acknowledges the gap directly: current conventions address LLM completions but lack coverage for full agentic systems. The foundation every major tracing tool builds on is still catching up to what practitioners already run in production.

What I keep turning over is whether this is an engineering problem or something more permanent. The conversation stays framed around cost efficiency and logging. Use the cheap model for classification, the expensive one for hard reasoning, and eventually better observability will sort out the governance questions. Maybe. And yet something here feels more structural than that. "The agent" has become a name for a composite whose reasoning character shifts at each routing decision. Better logging will tell you which model ran at each step. The authorization was still granted to a unified subject that doesn't exist in the trace, and no amount of after-the-fact reconstruction makes that coherent. I genuinely don't know if that's a gap waiting for better tooling or a condition that persists regardless. The paperwork still assumes the subject is real, and the routing table is already evidence that it isn't.