How Unify cut its AI agent costs 95% in two weeks

A CTO walks through the actual engineering behind a 90-95% agent cost reduction in two weeks: collapsing many sub-agents into one code-writing main agent, working around OpenAI's 15 req/s prompt-cache limit by hashing users across 16 buckets, and building a zero-cost TypeScript sandbox. It's rare to get this level of production detail on agent economics, including why they optimize tool-call efficiency over per-token price.

• Tool-call efficiency dominates cost: sub-cent LLM calls are noise next to $1+ data vendor API calls, so a 10x-cheaper model that makes 3x more tool calls is a bad trade.
• OpenAI prompt caching caps at ~15 requests/second per cache key; Unify distributes load by hashing user IDs across 16 buckets.
• When using LLM-as-judge or user simulation, use a different model family than your main agent to avoid mode collapse in evals.
• A robust upfront planning step that scouts trajectories and picks high-precision APIs first was one of the biggest efficiency wins.

Part of Issue Nº 005: How Unify cut agent costs 95%, and why RL-trained agents break in the wild