Measuring & Governing Token Spend

You can't optimize what you can't see. Build the observability, budgets, and governance that keep token costs optimized after the heroic cleanup ends.

The Visibility Gap

Most organizations discover AI overspend the way they once discovered cloud overspend: a shocking invoice, weeks later, with no way to tell which team or feature caused it. The fix is the same discipline cloud FinOps brought to compute - applied to tokens. Everything in this lesson exists so that the question "where is the money going?" has an instant, accurate answer.

Instrument Every Call

Log, for every model call:

• Tokens in / out - split by cached vs. uncached
• Model and reasoning/thinking budget used
• Feature / endpoint, and team or cost center
• Latency and success/failure
• For agents: run ID and step count, so runs can be costed end-to-end

Providers return exact token usage in every API response - capture it at your gateway or SDK wrapper so coverage is automatic, not per-team voluntary.

The Metrics That Matter

Budgets, Alerts, and Circuit Breakers

Reactive dashboards are not governance. Make limits structural:

• Per-team and per-feature budgets with alerts at 50/80/100% of the monthly mark.
• Hard per-run caps for agents (Lesson 3) - the difference between a logged failure and a billing incident.
• Anomaly alerts - today's spend ≥3× the trailing average fires a page, not an email.
• Circuit breakers - a runaway feature gets automatically downgraded to a cheaper model or rate-limited rather than taken down. Degrade gracefully; never choose between an outage and an invoice surprise.

Make Cost a Review-Time Concern

Optimized systems regress one merged PR at a time. Catch it where it happens:

• Add expected token cost to feature design templates - a simple "cost per 1,000 uses" estimate.
• Include a token-cost check in code review for AI features: model tier justified? max_tokens set? prompt structured for caching? reasoning budget appropriate?
• Run a monthly token review: top-10 features by spend, cache hit rates, routing share, and any regressions. Treat a cost regression exactly like a performance regression.

Chargeback and Accountability

Showback (publish per-team costs internally) changes behavior; chargeback (actually bill the team's budget) changes it faster. When the team that ships the feature owns its token bill, optimization becomes self-sustaining - engineers start asking "does this need the frontier model?" without being told to. That cultural end-state is the real goal of this lesson.