LLM cost management: budgets, alerts, per-team visibility

LLM spend goes wrong in two ways, and most companies meet both within the first year. The loud failure is the surprise total: an agent stuck in a retry loop over a weekend, and an invoice roughly triple the month before. The quiet failure is the total nobody can explain. Finance asks which teams spent the money, and the answer is that the provider bills one account, and that account has never heard of your org chart.

Both failures have the same fix, and it is not a spreadsheet.

What is LLM cost management?

LLM cost management is metering every LLM request as it happens, with attribution to a user, team, key and model; enforcing budgets in the request path; alerting a person before a limit trips; and keeping analytics that answer who spent what and why. All of it runs at one control point that every request crosses, usually an LLM gateway.

Why the provider invoice can’t do this

The natural instinct is to wait for the bill and then allocate it. For LLM traffic the bill is the wrong instrument. It aggregates per account, so teams and projects never existed in it. Each provider bills in its own token currency. A subscription seat looks the same whether it ran an agent all month or sat idle. And once one shared key serves five scripts, no forensics can separate them afterwards. An invoice is a settlement document. It closes the books on decisions made weeks earlier, and it was never designed to inform them.

Two levers sit outside this post. Prompt caching and context discipline are application-design decisions, and they can move a bill further than anything below. Both depend on the same prerequisite: until requests carry a key and a metered price, you cannot tell whether either one worked.

The five mechanics, in the order they work

The first three are strictly ordered: attribution, then metering, then enforcement. Attribution without metering is a naming scheme. Budgets without attribution are caps on numbers nobody owns. Alerts and analytics both hang off the meter and can arrive in either order.

1. Attribution: a virtual key for everything

Every person, application and agent gets its own virtual key. The real provider keys stay inside the gateway, encrypted at rest, and nobody gets a copy. A virtual key is shown once and cannot be recovered later, which has a useful side effect: issuing one is free, so two people never have a reason to share. It feels like bureaucracy, and it is what makes every later step possible.

2. Metering at request time, not at month-end

As each request crosses the gateway, the gateway counts its tokens, prices them against the provider route that actually served the request, and logs the result with the key that incurred it. One ledger, one currency, recorded while the request is still running rather than weeks later as a closed line item.

The invoice arrives six weeks after the decision that caused it, addressed to nobody in particular.

3. Budgets: which cap belongs at which level

Budgets exist at five scopes: key, team membership, team, user and organization, and each one is debited independently. Wherever you enter a figure, it is a hard cap. The gateway refuses the next request once the figure is reached. Requests already in flight finish, so a cap is a ceiling with a small bounded overshoot rather than a wall. Say so before you promise finance an exact number.

Which figure to enter depends on who the spend belongs to. A runaway script is a technical fault, so its own key gets a tight hard cap: the loop that started at 6 p.m. is refused before its next call goes upstream, not at invoice time. A team is a different case. Freezing forty people over one bad estimate punishes everyone, so a team’s cap should sit well above the plan as a safety net, and the alert threshold below it does the real work.

Two things are roadmap, not shipped: a Department tier between organization and team, and one-time grants for a migration week. Where budgets sit among the other controls, access, guardrails, logging and change control, is in LLM governance: a working framework.

4. Alerts that reach someone who can act

A budget warning sent to a platform admin about a marketing team’s spend produces a forwarded email. The team-supervisor role fixes that. It gives team-level visibility and its own alert threshold without platform admin rights, so the warning lands with the person who knows whether the spike is a launch or a bug, and how much headroom to leave is their call.

Alerts fire when spend crosses a configurable share of the budget, 70% by default, and arrive in chat, in the browser and by email. Thresholds are evaluated on a cycle, so a burst fast enough to cross the threshold and the cap at once still stops at the cap. A team envelope holds because someone owns it, not because the gateway enforces it. A team that crosses its threshold three cycles running is a conversation with a finance partner, and the spend history makes it a five-minute one.

5. Analytics that answer real questions

ControlTower shows usage as KPI cards with period-over-period deltas, model mix and top users, sliceable per user, team, key and model over time. The same data is queryable through a read-only analytics MCP that an admin can switch on. All of it lives in your own PostgreSQL, and every month of history makes the next budget conversation shorter.

Where to start, and in what order

Virtual keys for everyone in week one, because nothing else works without them and nobody notices the change. Metering runs itself from there. Hard caps go on unattended keys first: agents, scripts and CI, before they go anywhere near people. Team envelopes come last, after a month of real numbers, because an envelope set without them is a guess, and a guessed envelope is the one that freezes forty people.

What this buys is not a smaller bill. It is a bill that behaves like a utility: attributable, capped where it must be, and owned by whoever runs the team rather than by a procurement cycle. SecondStack has no per-seat licensing, so the spend is provider usage billed to your own accounts. The per-seat-versus-consumption math is in this post on ChatGPT Enterprise pricing.

None of this needs a FinOps team. It needs one control point that every request crosses, which is the gateway argument. Until you have one, every conversation about LLM cost is a conversation about a number nobody can take apart.