--- title: "Foundations" canonical: "https://current.tinyfish.ai/issue/7/foundations" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/7/foundations" issue: 7 section: "foundations" description: "Conceptual clarity earned from building at scale" authors: ["Nora Kaplan","Rina Takahashi"] published: 2025-12-14 updated: 2025-12-11 articles: 11 --- # TinyFish > CURRENT is a journal about AI agents in production, for the engineering and product leaders who have to make them work. Drawing on experience building enterprise web agents, it explains why the demo succeeded and the deployment didn't, and what that means for choosing vendors, briefing the board, and avoiding mistakes others have already made. [Read on the website]() ## Issue 7 — Foundations Conceptual clarity earned from building at scale ### What Building Teaches Builders - [When to Automate and When to Keep Humans in the Loop]() — By [Nora Kaplan]() · Published 2025-12-10 · Format: Field Guide Teams come to us after watching agent demos, asking: "Can we automate our entire competitive monitoring process?" Sure. Technically, we can. That's exactly when things fall apart. The agents work fine. Trust collapses. Pricing data gets extracted incorrectly after a site redesign, and suddenly the whole system feels unreliable. Operating web agents across thousands of sites taught us this: technical capability isn't the constraint. Knowing where to draw the automation boundary is. ### Tools & Techniques - [When Scrapers Break Clean]() — By [Rina Takahashi]() · Published 2025-12-10 · Format: Tools in Context Your scraper stops returning prices overnight. Every field comes back null, and your monitoring catches it immediately because the data structure broke. This is the kind of failure teams can work with—loud, obvious, fixable. Rule-based validation exists for these moments, stopping bad data before it flows downstream. At scale, catching structural breaks isn't optional. It's survival. - [When Data Drifts Quietly]() — By [Rina Takahashi]() · Published 2025-12-10 · Format: Tools in Context Your scraper runs perfectly for months. Every field validates, types match, formats check out. Then someone in analytics notices the "product weight" field now contains shipping estimates. The data structure is fine. The meaning has drifted. This is what rule-based validation misses—the quiet corruption where everything looks right but nothing means what it should anymore. ### Pattern Recognition - [Procurement Teams Can't Buy What They Can't Measure]() — Published 2025-12-10 Watch what happens when procurement departments try to evaluate AI agents. They pull out the same RFP templates they use for CRM systems. Uptime guarantees. Fixed feature lists. Predictable outputs. ### Questions Worth Asking The questions you ask when evaluating production systems reveal what you've learned the hard way. Experienced builders skip "can this work?" and ask questions that expose what breaks at scale, what's expensive to fix later, and what marketing materials conveniently omit. These questions predict whether your POC becomes production infrastructure or expensive homework. Whether your database choice lets you grow or forces a rebuild. Whether your monitoring explains what's wrong or just signals that something is wrong. The right questions cut through demos and documentation to what actually matters when systems face real users, real data, and real consequences. - **Can You Retrofit Explainability Later?** — [Source]() — Published 2025-12-10 Tag: Explainability Architecture Teams skip explainability during POCs, planning to add it before launch. This almost never works. Explainability requires architectural decisions from day one. Skip it now and you're building technical debt that gets exponentially harder to fix. Ask how you'll explain decisions to users before you write the first line of code. - **Does Your Database Scale Sideways?** — [Source]() — Published 2025-12-10 Tag: Infrastructure Scalability When you need more capacity, can you add servers or just bigger servers? Horizontal scaling determines whether you grow or rebuild from scratch. Many databases make adding machines nearly impossible. Your initial choice locks you into a scaling path that's expensive to escape. - **What's Your SLA When Outputs Are Probabilistic?** — [Source]() — Published 2025-12-10 Tag: AI SLAs Traditional SLAs cover uptime and latency. AI systems need agreements around accuracy, adaptation speed, and decision quality. Cloud providers focus on availability because quality degradation resists contractual definition. If your SLA only measures uptime, you're missing what actually matters for AI. - **Does Your Observability Explain Why?** — [Source]() — Published 2025-12-10 Tag: Production Observability Monitoring shows when something breaks. Observability shows what's happening and why it's happening. For agents, track context drift and reasoning patterns, not just errors and latency. Companies with structured evaluation frameworks see sixty percent fewer production incidents than those watching dashboards alone. - **How Different Is Production From Your Test Data?** — [Source]() — Published 2025-12-10 Tag: Evaluation Strategy Offline evaluation uses curated datasets in controlled environments. Online evaluation monitors real user interactions. The gap between them reveals model drift, unexpected queries, and edge cases your test data missed. Testing only offline means optimizing for conditions that don't exist in production. - **Can You See Cost Per Request Live?** — [Source]() — Published 2025-12-10 Tag: Cost Monitoring An agent calling an LLM five times might be justified quality improvement or a bug causing runaway costs. Real-time cost visibility reveals whether you're trading money for quality or bleeding budget on inefficiency. Without live monitoring, expensive mistakes show up in monthly bills, not deployment dashboards. ### In Dialogue With Complexity - [A Conversation with the Algorithm That Decides If You're Human Enough]() — Published 2025-12-10 We're in what Vee Three calls "the observation deck," though it's really just a conference room with too many monitors. Numbers flicker across every screen: 0.7, 0.3, 0.9, 0.1. Each one is Vee Three's verdict on whether someone, somewhere, is human enough. ## Other sections in Issue 7 - [Daily Brief]() - [Market Pulse]() - [Practitioner's Corner]() - [Echoes]() - [Vision]() ## More - [Publication guide]() - [Issue archive]()