--- title: "Practitioner's Corner" canonical: "https://current.tinyfish.ai/issue/1/practitioners-corner" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/1/practitioners-corner" issue: 1 section: "practitioners-corner" description: "Lessons from the field—what we see building at scale" authors: ["Rina Takahashi","Nora Kaplan"] published: 2025-10-28 updated: 2025-10-28 articles: 10 --- # TinyFish > CURRENT is a journal about AI agents in production, for the engineering and product leaders who have to make them work. Drawing on experience building enterprise web agents, it explains why the demo succeeded and the deployment didn't, and what that means for choosing vendors, briefing the board, and avoiding mistakes others have already made. [Read on the website]() ## Issue 1 — Practitioner's Corner Lessons from the field—what we see building at scale ### Dispatches from Production Scale - [The 13% Problem]() — By [Rina Takahashi]() · Published 2025-10-28 · Format: Operations Field Notes Been watching this pattern for months, and it's starting to feel like weather—small pressure systems that look harmless until they meet and create something bigger. Every ops team mentions the same thing: clusters running at 13%, maybe 15% utilization. Nobody can quite explain why. Not because anyone's doing something wrong. Everyone's doing the reasonable thing. Request a safety margin, protect your service, move on. ### Mapping the Invisible Web - [The Multiplying Web: When One URL Becomes Dozens]() — By [Rina Takahashi]() · Published 2025-10-28 · Format: Field Dispatch Funny how we talk about "checking a website" like it's singular. Spent years in ops watching people casually say that, not realizing they're actually looking at thirty different versions of the same URL serving completely different content depending on location. Not just language toggles—different products, different prices, different inventory. Same address, parallel realities. ### Theory Meets Production Reality - [The Optimization That Made Monitoring Impossible]() — By [Nora Kaplan]() · Published 2025-10-28 · Format: Parallax View Been tracing why competitive monitoring got so complicated—keeps coming back to personalization, which everyone loves as users but creates this weird blindspot. Platforms optimize for individual conversion, totally rational, except that same logic makes systematic monitoring nearly impossible. Every layer they add to improve your experience multiplies complexity for understanding the platform as a whole. Not that they're ignoring the side effects, more like—it's not their problem. Someone else's infrastructure challenge. Anyway, wanted to map the business logic first, then show what it does to monitoring operations. - [When "Check the Price" Means Check 48 Prices]() — By [Nora Kaplan]() · Published 2025-10-28 · Format: Parallax View After mapping why platforms personalize, needed to show what that means when you're actually trying to monitor them. Found this case—one hotel, "check the price" meant checking 48 versions of the same page. Different locations, devices, login states, keeps multiplying. Before the A/B tests even. Started documenting what teams actually build when they need consistent data from surfaces showing different content to everyone. The arithmetic gets absurd fast. Gap between "just check the website" and the infrastructure that takes—that's the whole operational reality nobody sees from the outside. ### Field Notes from the Ecosystem - **Cross-Cloud Dependencies Amplify Regional Outages** — [Source]() — Published 2025-10-28 Tag: Cascading Failure AWS us-east-1 went down October 20 (07:11-10:53 UTC). Incident.io, hosted entirely on Google Cloud, lost on-call notifications and SAML authentication anyway. Third-party dependencies crossed cloud boundaries. Their multi-region architecture couldn't help. Detection lag from AWS failure to internal paging: 10 minutes. - **Login Endpoints Get Aggressive Rate Limits** — [Source]() — Published 2025-10-28 Tag: Authentication Defense Better Auth framework ships with 100 requests per window for general traffic. Email login paths get 3 requests per 10 seconds. Rate limiting tracks via x-forwarded-for header, disabled in dev mode. Production authentication systems tighten the window where credential stuffing happens. - **Per-Consumer Isolation Replaces Shared Quotas** — [Source]() — Published 2025-10-28 Tag: Rate Limiting Rate limiting now isolates per API consumer: per key, per user account, per IP. One bad actor can't exhaust shared quotas. Malicious behavior becomes easier to identify. Observed threshold: 100 requests per minute for short-duration abuse prevention. Global limits are out. - **Postmortem Window Holds at 36-48 Hours** — [Source]() — Published 2025-10-28 Tag: Incident Response Formal postmortem review happens 24-48 hours after incident resolution, while details remain fresh. No expectation for immediate root cause investigation, especially for off-hours mitigation. Initial research often happens same-day. The 36-48 hour window for completion remains standard across the industry. - **Three-Layer Defense Without Traditional Authentication** — [Source]() — Published 2025-10-28 Tag: API Security WAF blocks known threats. Custom authorizer validates short-lived tokens proving app origin. Session-based rate limiting tracks per-token usage. This addresses CGNAT environments where IP-based limiting fails. Session tokens replace IP addresses as the primary rate limit identifier. No traditional authentication required. - **Dev Endpoints Remain Exposed Post-Deployment** — [Source]() — Published 2025-10-28 Tag: Shadow APIs Shadow APIs are endpoints created during development or testing that stay live in production. Organizations deploy API discovery tools to find unmanaged endpoints. Security testing integrates into DevSecOps pipelines. AI model inference APIs get targeted for training data extraction. The attack surface grows from forgotten test code. ## Other sections in Issue 1 - [Daily Brief]() - [Market Pulse]() - [Foundations]() - [Echoes]() - [Vision]() ## More - [Publication guide]() - [Issue archive]()