Controlling LLM Costs Before They Control Your Roadmap
AI features that work but cannot be afforded get switched off. Caching, model tiering, context discipline and per-tenant budgets — with the numbers that matter.
The most common way an AI feature dies is not poor quality. It is a finance review. Cost scales linearly with usage in a way most software does not, so a successful launch produces a bill that makes the feature look like a mistake. This is preventable, and cheaply, if you design for it early.
Instrument cost per feature on day one
Token spend attributed to a feature, a tenant and a user is the single most useful piece of telemetry in an AI system. Without it, cost is one undifferentiated number and every optimisation is guesswork. With it, you usually discover that one feature used by a handful of people accounts for most of the spend.
You cannot optimise a bill you cannot break down.
Cache more than you think you can
Identical requests are more common than intuition suggests, especially for embeddings, classification and retrieval over stable content. Cache at several levels: the embedding of a document that has not changed, the retrieval result for a repeated query, and the final response where the inputs are identical. Prompt caching offered by providers cuts the cost of long, stable system prompts substantially.
Tier your models honestly
Routing every request to the largest available model is the default and it is wasteful. Most pipelines contain steps — classification, routing, extraction, short summarisation — that a smaller model handles at a fraction of the cost with no perceptible quality difference. The evaluation harness is what lets you make this substitution safely rather than nervously.
Context is the silent cost driver
Teams optimise the model choice and then stuff 40,000 tokens of retrieved context into every call. Retrieval quality is a cost lever: better ranking means fewer chunks, and fewer chunks means a smaller bill and often a better answer. Trimming context is usually the highest-return optimisation available and it improves quality at the same time.
Budgets and circuit breakers
Per-user and per-tenant spend caps prevent one enthusiastic customer, one runaway loop or one abusive script from producing a five-figure surprise. Enforce them server-side with a defined behaviour when the cap is hit — degrade to a cheaper model, queue the work, or refuse clearly. Anything is better than silent unlimited spend.
Know your cost per outcome
Cost per request is an engineering metric. Cost per resolved ticket, per processed invoice, per qualified lead is a business metric, and it is the one that survives a finance review. When you can state that the system costs ₹9 per document against ₹120 of human handling time, the conversation stops being about the AI bill.
