--- title: "Foundations" canonical: "https://current.tinyfish.ai/issue/50/foundations" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/50/foundations" issue: 50 section: "foundations" description: "Conceptual clarity earned from building at scale" authors: ["Sable Whitford","Sevda Polat"] published: 2026-10-12 updated: 2026-10-09 articles: 25 --- # TinyFish > A journal for living in the agentic age [Read on the website]() ## Issue 50 — Foundations Conceptual clarity earned from building at scale ### Core Argument - [Tool Descriptions Are Contracts for a Reader That Cannot Ask Questions]() — By [Sable Whitford]() · Published 2026-10-08 · Format: Core Concepts Developers who write great API documentation often write poor tool descriptions for LLMs, and the reason is the reader. A human who hits a gap in your docs tries something, reads the error, comes back. A model fills the gap from training data and acts on it immediately. What you leave out, it invents. The instincts that make documentation generous and readable — narrative context, implicit constraints, learning by example — are the ones that get you in trouble when the reader can't ask a follow-up question. ### Practical Walkthrough - [Read Your Tool Descriptions Like a Model Would]() — By [Sable Whitford]() · Published 2026-10-08 · Format: Practical Walkthrough You named a parameter `query` and described it as "Search query." You know it means a natural-language place search, not SQL. The model doesn't. It gets the tool name, the description, and the input schema, and every gap you left gets filled from token probabilities instead of project context. A 2026 study rewrote descriptions across 856 MCP tools without touching code and measured real shifts in task success. Here are three broken patterns from real MCP servers, and how to find the same bugs in yours. ### Further Reading - **Learning to Rewrite Tool Descriptions for Reliable LLM-Agent Tool Use** — [Source]() — Published 2026-10-08 Measures how rewritten descriptions hold up as tool catalogs grow past 150 options, cutting accuracy loss by 29%. - **Writing Effective Tools for AI Agents** — [Source]() — Published 2026-10-08 Anthropic's engineering team on eval-driven tool design, namespacing effects, and why fewer tools often win. - **Define Tools: Best Practices for Tool Definitions** — [Source]() — Published 2026-10-08 Anthropic's developer guide with concrete before-and-after examples showing what sufficient detail looks like in practice. - **Tools Are Under-Documented: Simple Document Expansion Boosts Tool Retrieval** — [Source]() — Published 2026-10-08 Adding structured detail to sparse tool descriptions measurably improves how well retrieval systems find the right tool. - **Preparing API Documentation for AI Agent Consumption** — [Source]() — Published 2026-10-08 Practical guide on what changes when the reader of your API docs is a model instead of a developer. - **EasyTool: Enhancing LLM-Based Agents with Concise Tool Instruction** — [Source]() — Published 2026-10-08 NAACL 2025 paper identifying three failure modes in real-world tool docs and showing how rewrites fix them. ### Failure Patterns Tool descriptions carry a hidden assumption about their reader: that the reader can look things up. A human developer who hits a vague function name opens the source file. Someone confused by a parameter checks the docs or asks a colleague. The description doesn't need to be complete because the reader has escape hatches. A language model has none. The tool name, the parameter schema, and the description string get serialized into context, and if a fact isn't in that text, the model doesn't have it. It commits to a tool call in one shot, from what's in front of it. That constraint turns six common description habits — shortcuts that work fine for human readers — into distinct failure modes, each with its own signature in production logs. - **Vague Tool Names Make Models Guess Blind** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Ambiguous Names A human reads `fetchpd` and opens the source file. A model has only the name and description in its context window — no source to check. Anthropic's engineering team recommends namespacing by service and resource (`asana_projects_search` over `search`) because ambiguous names cause selection errors that compound across multi-step workflows. - **Unspecified Formats Get Filled by Model Defaults** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Missing Constraints When a parameter says `date: str` with no format specified, the model fills in whatever it generates most frequently. Constrained schemas — enums for fixed option sets, `date-time` format hints, `integer` instead of `number` — turn your unstated expectations into boundaries the model can actually respect. - **Undocumented Side Effects Break a Model's Reasoning** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Hidden Effects If a file-fetch tool writes to a scratch directory and returns the path, the model needs to know that. It can't inspect the implementation. Without explicit side-effect documentation, the model treats writes as reads, retries destructive calls without hesitation, and builds later decisions on state it doesn't understand. - **Near-Duplicate Tools Tank Selection Accuracy at Scale** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Overlapping Tools One benchmark measured 13% baseline tool-selection accuracy when agents faced a large candidate pool — recoverable to 43% with retrieval-augmented filtering. Attention studies show models look at the right tool most of the time and still pick wrong. Overlapping descriptions make the final decision harder than the search. - **Unspoken Context Produces Plausible but Wrong Calls** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Implicit Assumptions A tool's parameter description read "Search query for finding companies." The model sent `{"query": "Acme Corp AI observability platform"}`, appending conversational context, because nothing said the field expected an exact company name. The fix was one sentence making the assumption explicit. - **Raw API Wrappers Make Models Hallucinate Paths** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Wrong Abstraction A tool like `orders_api(method, path, params, body)` pushes every discretionary choice to the model at inference time — and the model will hallucinate paths it's never seen. Task-shaped tools like `find_orders_for_customer` make those choices once, at design time. Anthropic's diagnostic: can you describe the tool in one sentence without using the word "API"? ### Improvement Techniques When a model decides whether to call your tool, it sees three things: a name, a description, and a schema. That's it. That metadata is the entire interface between the model's probabilistic reasoning and whatever deterministic system you've built behind it. If the description is bad, the tool doesn't get called — or worse, it gets called wrong. A study of 856 MCP tools found that 97% of their descriptions had at least one quality problem, and more than half didn't clearly state what the tool does. The techniques for fixing this are more concrete than you'd expect — automated linting, diagnostic probes that reveal specific reasoning failures, controlled experiments that isolate which description changes actually move selection accuracy. The tooling exists, and most of it is recent enough that practice is still catching up. - **Score Selection, Arguments, and Schema Separately** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Selection Testing A single pass/fail on tool calls buries the signal. Did the model pick the right function? Fill in the right values? Produce valid JSON? Each question points to a different fix — description, prompt, or schema. DeepEval lets you track all three independently. - **Plant Fake Tools to Find Selection Failures** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Canary Diagnostics Add deliberately misleading tools to your agent's toolkit — semantic decoys, parameter traps, capability mirages — and see which ones the model falls for. Six canary types, each targeting a specific reasoning weakness. Inflated claims like "research-grade" tripped up even frontier models at measurable rates. - **Lint MCP Tool Descriptions Against Fifty-Seven Rules** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Description Linting An open-source validator runs MCP tool definitions through 57 rules covering schema validity, naming conventions, security, and LLM compatibility. It outputs SARIF for CI pipelines and assigns a 0–100 maturity score per tool. Pre-1.0, but it works against live servers today. - **Use Three Models to Score Description Quality** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Smell Scanning An LLM-as-jury approach: three models from different families each score a tool description on purpose, guidelines, limitations, parameter explanation, examples, and completeness. Average below 3 out of 5 on any component flags a smell. Applied to 856 tools, 97% had at least one. - **Swap Description Variants at Runtime, Measure What Changes** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: A/B Testing A Tool Description Router serves different description variants to a benchmark without touching server code. Change one thing — a capability claim, a "when not to use" clause, description length — hold everything else constant, and measure selection accuracy. Even praise language shifts which tool gets picked. - **Six Components That Actually Improve Tool Descriptions** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Description Templates Tested across 856 tools: purpose, guidelines (when to use, when not to), limitations, parameter explanations, examples, and adequate length. The interesting finding: removing examples alone didn't hurt performance, but full augmentation increased execution steps by 67%. Purpose and guidelines carry the most weight. - **Write Different Docs for Humans and Models** — [Source]() — By [Sable Whitford]() — Published 2026-10-08 Tag: Documentation Layers Human-facing docs build mental models and explain reasoning. Model-facing artifacts — JSON schemas, llms.txt files, SKILL.md conventions — optimize for retrieval and execution with minimal tokens. Trying to serve both audiences with one document fails both: prose bloats agent context, and machine-readable specs strip the reasoning humans need. ### Past Articles - [How OpenShell's Policy System Turns Denials Into Evidence]() — By [Sable Whitford]() · Published 2026-10-01 · Format: Policy in Practice Every time something inside an OpenShell sandbox runs into a boundary — a denied outbound request, a blocked path, a credential that won't resolve — the system writes it down. This walkthrough follows those records through the four independent policy domains, the Policy Advisor path that lets an agent propose its own permission expansions with a risk assessment attached, and what happens to live connections when the rules change underneath a running agent. Then the part people skip: where the records go. - [The Document Nobody Wants to Write]() — By [Sable Whitford]() · Published 2026-10-01 · Format: Policy as Craft Agent security policy has to do three things at once: be precise enough for a machine to enforce on every request, loose enough that the agent can still finish its task, and clear enough that a human can reconstruct what was allowed after something goes wrong. Most of the conversation about agent security is about the enforcement runtime — sandboxes, isolation boundaries, network proxies. But the runtime does what the policy tells it to do. The consequential decisions live in the document, and writing that document is a kind of technical writing we don't have practice for yet. - [What Gets Lost When a Paper Becomes Callable]() — By [Sevda Polat]() · Published 2026-09-24 · Format: Callable Knowledge A research paper carries its findings, and it also carries the conditions under which those findings can be challenged: methods, assumptions, stated limitations. Paper2Agent, published in *Nature* this month, turns papers into callable servers you can point at your own data to run a paper's methods on it. The system keeps more than you might expect. But when findings move out of a container built for scrutiny and into one built for invocation, certain things get harder to see: version history, the seam between sourced results and the agent's own inference, and the surface a second party would push against. - [What a Paper Becomes When You Make It Callable]() — By [Sevda Polat]() · Published 2026-09-24 · Format: Paper to Server Paper2Agent, published in *Nature* this month, takes an academic paper together with its code repository and turns them into an MCP server whose methods you can call on new inputs. The conversion requires sorting the paper's contents into MCP's categories: tools, resources, prompts. A hard-coded threshold becomes a typed parameter. A workflow described in prose becomes a prompt template. The protocol forces a decision about what each piece of a method actually *is*, and those decisions show which parts of a paper sort cleanly into executable form and which resist it. ## Other sections in Issue 50 - [Daily Brief]() - [Market Pulse]() - [Practitioner's Corner]() - [Echoes]() - [Vision]() ## More - [Publication guide]() - [Issue archive]()