--- title: "Practitioner's Corner" canonical: "https://current.tinyfish.ai/issue/14/practitioners-corner" publication: "TinyFish" publisher: "TinyFish" language: "en-US" guide: "https://current.tinyfish.ai/llms.txt" permalink: "https://current.tinyfish.ai/issue/14/practitioners-corner" issue: 14 section: "practitioners-corner" description: "Lessons from the field—what we see building at scale" authors: ["Nora Kaplan","Rina Takahashi"] published: 2026-02-02 updated: 2026-01-29 articles: 15 --- # TinyFish > CURRENT is a journal about AI agents in production, for the engineering and product leaders who have to make them work. Drawing on experience building enterprise web agents, it explains why the demo succeeded and the deployment didn't, and what that means for choosing vendors, briefing the board, and avoiding mistakes others have already made. [Read on the website]() ## Issue 14 — Practitioner's Corner Lessons from the field—what we see building at scale ### Dispatches from Production Scale - [The Calibration Work That Never Stops]() — By [Nora Kaplan]() · Published 2026-01-28 · Format: Field Dispatch Every Monday morning, a pricing analyst opens a spreadsheet containing 150 randomly selected hotel rates. Her job: verify that automated extraction matches what's actually on the websites. She's been doing this for six months. The sample size has dropped from 300 to 150—but it will never reach zero. Why does automation that works still require human checking? The answer reveals what enterprises actually spend money on when web agents move from proof-of-concept to production. ### The Builders - [Fabien Vauchelles and the 195 Regional Problems Hidden in 'Global' Web Automation]() — By [Rina Takahashi]() · Published 2026-01-28 · Format: Builder Profiles Your pricing agent runs perfectly in testing. You deploy globally. Twelve countries work fine. Three return mysterious session failures. At 3am you discover that what looked like authentication timeouts is actually how Singapore's CDN handles bot detection differently than Tokyo's. Same code, same provider, completely different regional behavior. ### The Number That Matters - [70% of Automation Failures Report Nothing Wrong]() — Published 2026-01-28 More than 70% of automation script failures are phantom failures. Tests report errors when the underlying functionality passed. A button moved three pixels. A modal animation took 50ms longer. The environment loaded fonts differently. The script breaks. The product works fine. ### Field Notes from the Ecosystem Network outages spiked 28% in a single week this January. API providers tightened quotas with days of notice. CDN exits triggered latency penalties measured in hundreds of milliseconds. Production systems surface what planning documents miss. The web's operational substrate keeps proving more fragile and constrained than infrastructure teams assume. When you're running agents at scale, these shifts stop being abstract industry news. They become Tuesday. Six observations from January. Each one a data point in the gap between how infrastructure is supposed to behave and how it actually performs under load. - **Outages Jumped 28% in One Week** — [Source]() — Published 2026-01-28 Tag: Network Infrastructure ThousandEyes logged 255 global network outages January 5-11, 2026. That's 28% above the prior week's 199 incidents. U.S. outages nearly doubled from 71 to 135. Public cloud network disruptions climbed 29% globally, with U.S. cloud outages up 54% week-over-week. - **Atlassian Switches to Points-Based Rate Limits** — [Source]() — Published 2026-01-28 Tag: API Constraints Effective March 2, 2026, Atlassian enforces points-based API rate limits for Jira and Confluence Cloud apps. Each request consumes points based on computational work performed, not simple request-per-second caps. Data returned and operations triggered now determine consumption. Most apps default to Tier 1. - **USPS Caps New APIs at 60 Hourly** — [Source]() — Published 2026-01-28 Tag: API Constraints USPS retired Web Tools on January 25, 2026. Replacement APIs cap at 60 calls per hour per API. Bulk address verification, CRM onboarding, real-time address capture all hit immediate throttling. For businesses processing addresses at scale, this creates operational bottlenecks. - **Google Halved Gemini Free Tier RPM** — [Source]() — Published 2026-01-28 Tag: API Quotas December 7, 2025: Google cut Gemini API free tier from 10 to 5 RPM for Gemini 2.0 Flash. Minimal advance notice. Working applications suddenly returned 429 errors. Google cited "sustainable service quality" as adoption exceeded projections. Developers scrambled to adjust. - **Akamai China Exit Adds 150-300ms Latency** — [Source]() — Published 2026-01-28 Tag: CDN Infrastructure Akamai exits mainland China CDN by July 2026. After June 30, traffic reroutes to Hong Kong, Singapore, Tokyo. Adobe Experience Manager platforms face estimated 150-300ms delays. Beijing users could see Time to First Byte exceed one full second. - **Discord Vendor Breach Hit 70,000 Users** — [Source]() — Published 2026-01-28 Tag: Vendor Security Discord's external vendor 5CA, handling customer support operations, experienced a security incident affecting roughly 70,000 users who contacted Support or Trust & Safety teams. Some leaked data included government ID images. Discord's internal infrastructure remained uncompromised throughout. ### Practitioner Resources - **YO's $3.7M Swap: Configuration Edge Cases** — [Source]() — Published 2026-01-28 Safeguards built for one context failed in adjacent systems. The validation gap between components cost real money. - **Microsoft Maia 200: Memory Hierarchy Under Load** — [Source]() — Published 2026-01-28 272 MB on-die SRAM with software-managed tiers. Data movement architecture determines whether compute tiles stay fed at inference scale. - **AWS: What Actually Breaks in Distributed Systems** — [Source]() — Published 2026-01-28 Independent failures and nondeterminism cause most impact. Edge cases vastly outnumber expected scenarios, making testing perpetually vexing. - **Operating Uber Payments: Sustained Operational Discipline** — [Source]() — Published 2026-01-28 Robust systems emerge from continuous iteration after incidents. Failover drills and monitoring rigor require organizational investment that never stops. - **Observability: Now Baseline, Not Aspiration** — [Source]() — Published 2026-01-28 Common telemetry standards across environments expected by default. Distributed infrastructure only becomes manageable at enterprise scale with this foundation. - **When to Reduce System Complexity** — [Source]() — Published 2026-01-28 Google and Amazon engineers discuss simplification at scale. Real-world problems are always the ones you didn't expect. ## Other sections in Issue 14 - [Daily Brief]() - [Market Pulse]() - [Foundations]() - [Echoes]() - [Vision]() ## More - [Publication guide]() - [Issue archive]()