Goldman Sachs deployed Claude across 12,000+ developers, reporting 30% faster client onboarding and 20% productivity gains. The numbers sound impressive until you ask what they measure.
"Developer productivity" typically tracks code generation speed in isolation, not whether the code ships faster or works better. A controlled trial found developers using AI tools took 19% longer to complete tasks while estimating they were 20% faster. Meanwhile, "30% faster onboarding" measures time-to-activate but ignores abandonment rates, error correction, or downstream service calls.
Goldman's CIO describes these agents as "early stages." They assist with coding and compliance work, but humans still define specifications and regulatory parameters. Speed without context is just noise.